At the time, I was working for a managed service provider, and a manufacturing client had gotten malware on their network that had encrypted their mapped network drives. What made matters worse was that the ransomware had spread to multiple computers. Back then, finding the source of infection wasn’t too hard. The payment instruction files on network drives had ownership attributes in NTFS that could tell you which username had created it, and the computers infected soon popped up error screens to prompt for that payment. Recovery wasn’t so bad either, as backups in those days were written to tapes that were out-of-band of the infection. It was a lot of work and time to get everything back to normal, but we didn’t have to pay a ransom, and we learned how to prevent this from happening again.
We have come a long way as an industry in the decade since. Users have greater cybersecurity awareness, and system hardening is a broad practice. Compliance frameworks have pushed best practices out in several verticals, and business leaders have a personal stake in cybersecurity.
I would never have imagined then that we would still be fighting ransomware. Tools like web filtering, email click protection, and software restriction policies worked well for years in reducing ransomware infections. But despite broad progress among early adopters, many laggards had not implemented sufficient processes to prevent ransomware, and worse yet, those organizations had inadequate backups and incident response plans to rely on when needed.
As a result, ransomware is a booming industry. The criminals continue to do this because they are making money hand over fist. Ransomware thieves made an estimated $20 billion in 2022. The reason why they are making so much money? Our negligence as an industry in implementing appropriate preventative and recovery measures. Nobody pays a ransom on purpose. They do it because they got caught flat-footed without proper backups and incident response plans to recover effectively and efficiently.
With all the money at stake, the cyber gangs are operating more like businesses than individuals. These are not disaffected teenage hackers operating out of their parent’s basement. They are professionals. Ransomware is their day job, and they operate with impunity, often from nation-states that have no interest in cooperating with Western law enforcement. Law enforcement, for that matter, is not of much help with respect to ransomware (with the notable exception of if you happen to operate a gas pipeline).
The ransomware that propagates on today’s networks is increasingly sophisticated. Their methods of propagation now work more like viruses or worms, exploiting vulnerabilities in corporate networks, as opposed to relying on unwary users to click on bad links (though the latter is still a common vector of infection). The methods of encryption are likewise stronger. Ransomware works to encrypt quickly so that maximum damage can be done before it is detected.
In the past, some organizations lucked out because the ransomware had flaws. Some used fixed decryption keys or poor algorithms that could be decrypted with some help on the Internet. Attackers are using strong encryption and embedding trojan horses to remain within networks even if the encrypted files are decrypted.
As a result, it is more critical than ever that IT organizations protect themselves with a dual focus on both prevention and recovery from ransomware. Conveniently, these steps have added benefits beyond just the threat from ransomware. Good cybersecurity controls can help broadly prevent current and emerging threats, while reliable backups are critical for many situations you can’t predict.
Over the last decade in IT, working both in managed services and professional services, I’ve seen over fifty different cases of ransomware ranging from the most minor of inconveniences to complete data loss and environment reconstitution requiring weeks of repair. What follows are five actionable steps that you can take now to dramatically reduce your risk of infection and improve your ability to repair.
One last thing before we get to ‘the good stuff.’ You should never, ever pay a ransom. These threats persist because it’s economically feasible for the bad guys to keep doing this. If we all stopped paying, there would be no more ransomware. A sobering study by CyberReason found that in many cases (6 out of 10 times), you won’t get your data back even if you pay, and even then, there is a large chance of being re-infected. 80% of organizations that paid a ransom were hit a second time, and in half of those cases, it was by the same initial attackers!
Have and maintain a solid backup solution
While I’d rather start by telling you how to prevent ransomware in the first place. The bottom line is this: you must have good backups that are tested on a regular basis. It is the most critical function of IT and, sadly, often overlooked. People take a set-and-forget approach to backups at their own expense. Having good backups can be broken down into four primary responsibilities:
1. Appropriate backup selections and cadence
Backups need to be running frequently. They must protect the right data (usually all servers and attached storage) and with enough retention to facilitate good restoration points before infection. Having a grandfather/father/son model for backups is an effective way to have a good combination of daily, monthly, and yearly backup points from which to recover.
Regular review and maintenance. This is so critical and yet so easy to overlook. There must be a designated responsibility for the success of backup jobs. Not only should backup systems automatically create cases when jobs fail to run, but someone must be responsible for manually reviewing and correcting backup failures regularly (Monday through Friday at a minimum). This involves proactively looking at backup consoles, not just awaiting job failure notifications (ask me how I learned that lesson).
Test your backups regularly. Backups are only good if you can recover from them. You should also understand how long recovery can take. If you must do a full restore on a giant storage volume, restoration could be measured in days rather than hours. Note that likewise, if you have a large server environment compromised, that restoration can quickly exceed what the organization considers reasonable. In those cases, a true disaster recovery solution is recommended for faster recovery.
2. Regular review and maintenance
This is so critical and yet so easy to overlook. There must be a designated responsibility for the success of backup jobs. Not only should backup systems automatically create cases when jobs fail to run, but someone must be responsible for manually reviewing and correcting backup failures regularly (Monday through Friday at a minimum). This involves proactively looking at backup consoles, not just awaiting job failure notifications (ask me how I learned that lesson).
3. Segment your backup infrastructure and have immutable backups
Backups do you no good if they too, become encrypted. Your backup system should be segmented from your production network with granular access controls to limit the spread of any infection and protect your backups. Using a third-party backup-as-a-service (BaaS) provider is one way of accomplishing this if you can’t easily do it yourself. Having backups be immutable means that they cannot be modified or overwritten for a period aligned with your retention. There are multiple approaches based on your backup solution, but this should be feasible with any modern backup solution.
4. Test your backups regularly
Backups are only good if you can recover from them. You should also understand how long recovery can take. If you must do a full restore on a giant storage volume, restoration could be measured in days rather than hours. Likewise, if you have a large server environment compromised, restoration can quickly exceed what the organization considers reasonable. In those cases, a true disaster recovery solution is recommended for faster recovery.
Some of these steps may take a little more effort, but it costs nothing to review your backups today. Ask your team to provide an update on the current health of your backups, or do it yourself and report to your leadership on the state of the system. If you are lacking resources, you can use this opportunity to make a case to your organization for what you need and what the consequences could be if you can’t close these gaps.
Action Step: Assign someone to review and report (within a week) on the efficacy of backups. A good report should include the count of systems protected (to be validated against your server and storage inventory), when the last good backup was made for each protected system, and how many recovery points are available for those systems. Then, turn this into a regular process with regular test restorations from a sample of systems.
Allow only trusted executables
Much like firewalls operate with a default-deny rule, restricting executables that run in your environment to only those that are trusted is one of the best bang-for-buck approaches you can take. By only allowing known-good executables, you can prevent many types of malware that may otherwise slip past your endpoint protection software.
Like any security measure, there is a trade-off between convenience and security, but the rewards are well worth the administrative overhead. There are several ways to implement an application allow-list, depending on the tools available to you. The first place to start would be with your endpoint protection solution if it has these capabilities. Alternatively, you can leverage Windows Defender AppLocker, or for older versions of Windows, you can use Software Restriction Policies (An AD group policy) to allow only executables signed by known trusted publishers (like Microsoft, Adobe, etc.) Note that application allow-listing isn’t just for workstations. You can and should use this on servers, too.
Action Step: Create a project to implement application allow-listing and resource it. Set a reasonable deadline to first pilot within 30 days and then implement more broadly across your environment within 90 days.
Perform regular vulnerability management
As mentioned earlier, modern ransomware no longer relies on users to click bad links or open attachments. Ransomware is now spreading across networks, leveraging software vulnerabilities to propagate like worms. If you want to test this theory, you can just open an RDS server to the public Internet and wait to see what happens. On a side note: If you have an RDS server exposed, please stop this right away and put it behind a firewall, an RDS gateway, or use Citrix or VMware to broker access, or you’re just begging for an infection.
Because new vulnerabilities are continually found, it is important to have a regular cadence of vulnerability scanning of your environment from inside and outside your network perimeter.
Of course, the scanning and identification of vulnerabilities are only meaningful if you take the right actions to address found vulnerabilities. At the same time, many can be addressed through good patch management (noted in step four below). Some vulnerabilities are the result of configurations (or misconfigurations) that patch management alone will not address.
One of the most familiar challenges I see in vulnerability management is what to do with legacy systems that have reached an end-of-life or end-of-support state. These systems often cannot be patched and usually cannot be retired for some purpose or another. There are two approaches to such cases.
1. One is to look for alternative ways to meet archival requirements while decommissioning legacy systems, such as extracting data into other formats (PDF or CSV files on network shares or migrating database tables into data warehouses) for later retrieval rather than relying on outdated client and server to access the application.
2. if you are truly stuck with a legacy system, the other approach is to segment the heck out of it. Put these systems in their networks with granular network controls and limit access both through only the network protocols needed but also limit them to trusted good hosts like an RDS or Citrix server.
Note that if your organization is developing its own software, you must also consider vulnerabilities within those applications. Your dev team should be embedding secure coding practices and vulnerability management within their dev processes.
Action Step: Perform a vulnerability scan of your infrastructure and assign the remediation of vulnerabilities with a deadline based on their criticality. If you cannot purchase the tools to do this, you can hire a cybersecurity firm to run them for you.
Effectively manage patches across your environments
Another step you or your team can take right now today is to look at the health of your patch management systems (and the policies that enforce them). You may leverage a third-party patch management system for many corporate environments. Is it working as expected? Many organizations find that their endpoint management solutions are not always working as desired or have the proper scope across systems. Sometimes, patches are stuck pending manual approval before being installed.
You need to think holistically about the enforcement of patch management, the approval of patches, and the schedule of updates. You also must make sure that everything is working end-to-end and that the patches are being applied. As mentioned in step 3 above, vulnerability scanning often tells me a lot about the success of an organization’s patching.
Note that patch management is not just for operating systems. Applications, hypervisor, and device-level patching are also often overlooked and can have disastrous consequences if missed. You need to think about the types of systems within your environment and ensure they have appropriate patching as well. Some 3rd party patches may not be feasible to manage centrally, and recurring tasks will be needed to ensure that one-off equipment such as firewalls, load balancers, printers, or SANs (Storage Area Network) are getting their patches regularly as well.
The bottom line is that patch management requires regular care and feeding, even with the best-automated systems. Someone needs to have clear responsibility and accountability for reviewing the health of patches. I suggest using automated cases in your incident management (ticketing) system so that you can ensure that patch management systems are being reviewed and that 3rd party patching is done on an appropriate periodic basis.
Action Step: Assign someone to review and report (within a week) on the efficacy of patches across OS, applications, and devices. Then, set up a recurring process to have regular checks.
Get help from an expert
Are you overwhelmed yet? These tips are just a handful of actionable ideas to reduce risk, but they can be a lot of work. Getting to a suitable cybersecurity posture can be arduous, and maintaining that state is its own challenge. If you don’t have the right resources, expertise, or tools to do so, I recommend seeking expert assistance.
I have seen some IT departments struggling with a lack of resources, feel overwhelmed, and become fatalistic about security. They feel that they are stuck and cannot get what they need from their organization to improve. Often, this is due to internal politics and the relationship between IT and finance teams or executives. Having an outside voice to help represent and quantify the risks an organization is facing can help dramatically to reduce these obstacles and help drive action to make needed investments.
Action Step: Contact us if you need help or find another reputable, well-established provider with certified cybersecurity experts.
Don’t become a statistic. Act now!
I’ve given you five steps you can start with today, most with little to no cost to get started. Some of these may even help you get more budget from your organization. Taking these steps can help protect you from ever having to pay a ransom and can help make the world a better place for us all. Seriously, what are you waiting for?
