Column One of the great threats to our civilization is space weather. Specifically, the Sun’s proven ability to target the planet with a tremendous cosmic belch of radiation, knocking out satellites, power grids, and networks worldwide.
In that context, SolarWinds’ choice of company name seems gruesomely apt. We still don’t know the full harm done by Sunburst, the splendidly evil hack of its Orion network monitoring platform, but it was global in scope, deep in reach, and hit only the highest-value assets. For months, the internal networks of government, military, and agency were compromised.
From the quality of the threat design, the range of techniques used, and the nature of its victims, this was a nation state at work and in MO and capabilities most likely Russia. It revealed a very good knowledge of not only the fabric of modern IT infrastructure, but the psychology of those who develop for and maintain it. Beautifully obfuscated, delicate in its use of steganography and layers of diversion. Sunburst will trigger another round in the arms race between hackers and opsec researchers.
Perhaps the most chilling aspect of the attack was how it propagated itself by installing itself as part of SolarWinds’ standard distribution and update system. This is a very old trick – anecdotally, mainframes in the 1960s were compromised by carefully faked system patch tapes sent to companies by mail – but of course rendered much more powerful by the automation and patch-quickly culture of today’s IT.
At the time of writing, it’s not clear whether the compromised .dll at the heart of the hack was built on SolarWinds’ own servers using the company’s own source, or whether a trojanised version was somehow signed and uploaded. As with so many complex infrastructure compromises, it doesn’t really matter and knowing the answer won’t do much to help understand the scope of the attack or the damage done. It does highlight, however, the central problem of trust.
There is no defence customers can deploy against a compromised vendor or supply chain that delivers what looks like legitimate code or services. Checksums and hashes only work if the reference isn’t despoiled; signed code is merely a special case of that concept of a chain of trust. Open source is more resilient in proportion to the size and interest of its community, but by no means immune.
Supply-chain attacks are insanely tempting to bad actors because they act like tapeworms, endoparasites that survive in their hosts because they can deactivate parts of the immune system. The downside to the worm is that if you knock out that part of their defences, which you can with a single dose of the right medicine, they have no further defences. Is there a one-dose pill for supply-chain attacks?
It is possible to create arbitrarily complex internal checks that what you’re shipping to your customers is what you think it is. CI/CD pipelines with their devoperatic test suites don’t in general retest code once it’s built, certified, and deployed, and their automation and high volume of updates pushed live create a high bandwidth channel to the customer base that is hard to monitor for subversion. Efficiency becomes a weapon in the hands of an enemy.
But a parallel pipeline that rebuilds everything continually and checks against the live files, with significant isolation from the production network and a second-pair-of-eyeballs policy for checking files in, could be made quite resilient to external attack. Not immune, but the idea is to build not just strong checks but ones that are themselves strongly defended. It’s parallel so it won’t impact on CI/CD efficiency, automated so the configuration can easily track the live system, and the whole thing not ridiculously resource-hungry.
Is that approach proportionate? Is it itself robust against attack? Would it catch the sort of mishap – SolarWinds build system FTP credentials being published in a repo – that may have led to Sunburst being opportunistically created? Does a company where that happens have bigger problems? Supply your own answers.
There’s little doubt, though, that the growth of automated, continual, distribution and patch systems bring up security problems of their own that in today’s very dynamic, adversarial infosec environment need to be considered afresh. Not just in how to fight the last war, but in designing for resilience from the outset. How to model the chain of trust and verification, internal to the vendor and in their complete customer community. How to balance the move towards frictionless continuous deployment with security that itself is continuous and frictionless. How to build security that isn’t just layered and deep, but redundant and self-checking.
At every stage in our evolution as a data-driven culture, we’ve had to evolve new security ideas – and usually, because we’re like that, after some disaster. The best aspect of Sunburst, which will become apparent over time, is that it is a highly evolved real disaster of substantial impact. Those of us on the side of the angels have to take this chance to evolve ourselves. ®