Once again, a supply chain attack is in the news; this time, it’s a ransomware attack against Kaseya which has impacted hundreds if not thousands of businesses. According to Kevin Beaumont, the attackers used a 0day vulnerability in the Kaseya VSA appliance to deploy a fake update to all systems it managed; that update is actually the REvil ransomware. As this is a VSA is used by Managed Service Providers (MSPs), this resulted in an attack not just on the MSPs but also their customers.

This event raises an interesting question on update mechanisms and how they play into supply chain attacks. This is what we’ll be discussing today.

Update mechanisms have been leveraged by attackers for many years and have, in some cases, had a substantial impact:

• The Passwordstate attack used updates to insert malicious code that stole credentials.
• The attack on SolarWinds pushed malware via updates to steal data from a variety of organizations, including the U.S. government.
• NotPetya resulted in massive damage and was deployed via a backdoored version of M.E.Doc, once again delivered as an update.
• Flame, perhaps the most famous example of this type of attack, leveraged the Windows Update system to deploy its malware.

These are but a few notable examples of incidents where an update mechanism is used to deploy some form of malware. Given this, one has to ask, are updates and especially auto-update mechanisms even safe to use?

Update mechanisms, automatic or otherwise, exists for a couple of primary reasons:

• Ensuring that users are on a recent version, if not the latest. This allows developers to focus their efforts only on newer versions, which simplifies support, reduces the need to deal with compatibility, and gives users the latest and greatest features quickly.
• Ensuring that users have the latest security fixes and improvements installed, to reduce the risk of being attacked by a known issue and provide additional protections.

It’s a well-understood fact that users aren’t good at manually installing updates, even when there are important security fixes. As such, there is a high likelihood that users will be rather behind the most recent version, and this is where automatic updates come in. By updating automatically, users don’t have to think about it, security teams don’t have to worry about it, and users get the best protection with the least effort.

The importance of updates should be clear to everyone, and the use of automatic updates plays an essential role in getting these updates deployed in a timely manner.

Before going on, I’d like to digress for a moment to address something that is too often overlooked. When most people think of a security update, they assume it fixes a vulnerability that was discovered — though that’s not always the case.

Updates may include additional hardening, which doesn’t address a known vulnerability, but instead adds extra protection to prevent exploitation or mitigate unknown vulnerabilities. This is more important than many people realize, as in some cases, these hardening improvements can block entire classes of vulnerabilities.

Such improvements don’t have a CVE, as they don’t represent a flaw, but without them, there may be an additional risk. Too often, these updates are skipped in manual update systems because it’s not clear enough what it does.

Another issue that is overlooked is how often vulnerabilities that are unknown or unreported are fixed unintentionally during refactoring or other code changes. Some updates may contain important security fixes; it’s just that the developer doesn’t know it.

One of the advantages of always using the latest version is that it increases the odds of these accidental fixes.

This fact may be surprising to those that don’t regularly work in vulnerability research, but it’s more common than one would expect.

To implement any update mechanism, and especially an auto-update mechanism, it’s vital that the infrastructure be hardened as much as possible, and the fewest number of people possible have access to it. There are few assets in a development shop more critical to protecting users than this infrastructure.

Servers must be locked down, psychical access restricted, code carefully reviewed, and operating system & software fully up to date.

Penetration testing should be conducted regularly, of not only the update infrastructure itself, but the entire system from end to end. For example, access to version control systems, build systems, signing systems, deployment & update systems, update notification, and update installation. Every step needs to be carefully reviewed and tested to ensure that there’s minimal risk of an attacker being able to influence what’s released.

The challenge now is this, how do the risks compare to the rewards? While this could be a case for a complete threat modeling exercise, I believe that there is an easier way to analyze this.

While some notable examples were listed above, the reality is that these attacks, while happening at an increasing pace, are still quite uncommon. After all, would we be having this discussion otherwise? Moreover, given how often auto-updating works, and the fact that with proper controls in place, the odds of a successful attack are small, there is a particularly strong argument that the rewards do indeed outweigh the risks involved.

While no system can achieve 100% security (give or take), we can achieve a high level of security by following the proper practices. It’s cutting corners and failing to properly invest in security that often leads to successful attacks against these mechanisms. While it’s true there’s risk, in reality, far more attacks are prevented than triggered.