Shadow Brokers, Equation Group, Oh My…

NSA / Equation Group

Yet again, a group known as The Shadow Brokers is in the news, with yet another leak from what is widely accepted as the NSA (Equation Group1 in APT terms). This release is, to many, the most important release of this leaked stolen material from the most elite and secretive hacking operation in the world.

This is a collection of a few notes on this highly unusual operation. If you haven’t read this excellent overview of the most recent release by Dan Goodin, you should do that now.

A Brief Timeline

The Shadow Brokers have released material on five occasions2:

August 13, 2016: Equation Group Cyber Weapons Auction – Invitation

A group of files were uploaded to Github (and quickly removed), including a file (“eqgrp-auction-file.tar.xz.gpg”) containing exploits and implants for Linux and similar operating systems, and a file (“eqgrp-free-file.tar.xz.gpg” with a password of theequationgroup) containing exploits and implants for a variety of networking equipment; the latter used as a sample.

October 20, 2016: Message#5 — Trick or Treat?

A second, smaller sample file named “trickortreat.tar.xz.gpg” with a password of payus.

January 13, 2017: MESSAGE FINALE

In what was purported at the time to be their final message, Shadow Brokers released a new file, “equation_drug.tar.xz.gpg” that included files that had been identified by Kaspersky AntiVirus as being Trojan.Win32.EquationDrug.

April 8, 2017: Don’t Forget Your Base

In a only somewhat surprising move, they posted a rambling treatise on US politics and Donald Trump, which included the password (CrDj"(;Va.*NdlnzB9M?@K2)#>deB7mN) for “eqgrp-auction-file.tar.xz.gpg” – the file they tried and failed to sell when they started. This contain the Linux exploits and implants that had been promised, though much of the content was rather old and of little interest.

April 14, 2017: Lost in Translation

Perhaps the most exciting release, as it included the Windows exploits and the Equation Group’s equivalent of Metasploit called FuzzBunch – but the story doesn’t end there, there was a big surprise included. The three files are “windows.tar.xz.gpg“, “odd.tar.xz.gpg“, and “swift.tar.xz.gpg” – with the last containing an unexpected surprise.

Burning Gold

As the community works to analyze the latest dump, going through the exploits trying to identify what they are, and if they are known some very interesting things have been found. ETERNALBLUE appears to be an 0day against Windows XP, Vista, 2003, 2008, 2008R2, and Windows 7 (it’s being reported that Windows 10 is vulnerable as well3) – one can imagine that there is panic in Redmond as Microsoft works to analyze it and prepare a patch. This is just one exploit, there’s also ETERNALCHAMPION, EASYBEE, EASYPI, ECLIPSEDWING, EDUCATEDSCHOLAR, EMERALDTHREAD, EMPHASISMINE, ENGLISHMANSDENTIST, ERRATICGOPHER, ESKIMOROLL, ESTEEMAUDIT, ETERNALROMANCE, ETERNALSYNERGY, EWOKFRENZY, EXPLODINGCAN, and ZIPPYBEER – there’s a lot of work to be done to fully understand the impact and figure out what needs to be fixed. A number of versions of Windows targeted by these exploits are not supported, such as Windows XP, Windows 2003, and Vista – meaning that there’s no patch coming, these systems will remain vulnerable forever.

What’s clear here is, there is some real value in the exploits that have been released (estimated at $2M or more) – and are likely very important to NSA intelligence operations (or at least they would have been till they learned that they had been compromised).

As NSA doesn’t talk about anything if it can be avoided, it’s unlikely that we will ever know what the impact is to their operations. We will likely see just how effective these exploits are though, as criminals work to leverage these exploits in exploit-kits and the like – you can be sure, just because these exploits are known, this certainly isn’t the last we’ll see of them.

The SWIFT Files

While it was known that they had various Windows exploits, they dropped something that is of less interest from a technical perspective, though fascinating from an operational perspective: there’s a collection of operator’s notes relating to attacks on SWIFT. These files include detailed system configurations, passwords, and step by step notes of what was done on the devices that were attacked.

Matt Suiche has done a great job of documenting what’s exposed by this cache, so I won’t repeat that here. If you want a better understanding of how they work, and what they’ve done, I highly suggest reading it.

The Origin

There’s been a great deal of debate about the source of these files, some have suggested that it was an insider, possibly even Harold Thomas Martin, though an insider makes little sense. The data is most likely from a jump-server – a server that NSA operators would push their files to, and connect to targets from. This would explain the type of files and documentation found – and would explain the files that aren’t included, such as source code, training material, and similar files that an insider would have access to, but wouldn’t be stored on a server outside of their network.

One likely scenarios is that a jump-server was captured by another intelligence agency, and the leaks and bizarre rants were part of a political play – though their choices of how to release information has greatly reduced their effectiveness. The SWIFT files could have been positioned as a WikiLeaks-styled bombshell, though was dropped quietly without the fanfare to make news outside of the normal technology publications – while the exploits have a substantial impact, they are of little interest to most people outside of the industry, the SWIFT work on the other hand, could be of much larger significance.

The level of detail about targets, and details from operator logs should allow the NSA to narrow the possible sources; I’m hoping that at some point there’s an official statement about who they believe is releasing these files – though the odds of that happening don’t seem good.


  1. The name Equation Group was coined by Kaspersky, while they didn’t directly state that Equation Group is the NSA (or possibly, just NSA’s TAO group), a close look at the evidence is quite clear. The material released fits with both the signatures of the Equation Group and with a US Government operation. There is some small chance that the attribution is wrong, but the odds of that appear to be quite low. 
  2. From the start of these releases, I’ve maintained an archive of the files, so that they are searchable and browsable. This has made it easy to review the files, and quickly find related files. It’s interesting to say the least. 
  3. Update 2017-04-15: Microsoft has released an update on their review of these exploits; ETERNALBLUE, ETERNALROMANCE, and ETERNALSYNERGY were all addressed in MS17-010 released in March 2017 (the source of the report was not listed). They confirmed the issue impacted Windows 10, and pretty much every other supported version of Windows. 

YAWAST 0.5 Released

Today, I’ve released the latest version of YAWAST, a security scanner for web applications that provides basic information about the application, and performs common checks so that you can move on to the fun part of testing more quickly. YAWAST also remains the only tool I’ve found that can perform an accurate test for SWEET32.

Here is the change log for version 0.5.0:

  • #35 – Add check for SameSite cookie attribute
  • #53 – Added checks for .well-known URLs
  • #75 – Use internal SSL scanner for non-standard ports
  • #84 – Improve the display of ct_precert_scts
  • #86 – Add check for Tomcat Manager & common passwords
  • #87 – Tomcat version detection via invalid HTTP verb
  • #88 – Add IP Network Info via api.iptoasn.com
  • #90 – Add HSTS Preload check via HSTSPreload.com
  • #91 – Enhanced file search
  • #96 – Scan for known SRV DNS Records
  • #97 – Search for Common Subdomains
  • #100 – Check for missing cipher suite support
  • #102 – Use SSLShake to power cipher suite enumeration
  • #76 – Bug: Handle error for OpenSSL version support error
  • #98 – Bug: SWEET32 Test Fails if 3DES Not Support By Latest Server Supported TLS Version
  • #99 – Bug: Cloudflare SWEET32 False Positive
  • #101 – Bug: SWEET32 False Negative
  • #103 – Bug: Scan fails if HEAD isn’t supported
  • Various code and other improvements.

TLS Certificates from the Top Million Sites

Thanks to the recent WoSign / StartCom issues with misused or flawed certificates, there was a question about how many of the certificates issued were actually active – so this seemed to be a good time to run a scan against the Alexa top 1 million sites to see what I could find.

Scan information: The scan was ran with an 8 second timeout – so any server that couldn’t complete a handshake within 8 seconds isn’t counted. The scanner attempted to connect to the domain on port 443, and if that failed, then attempted to connect to the “www” subdomain. No certificate validation was performed. The scan didn’t attempt any other ports or subdomains, so it’s far from a complete picture.

Of the top 1 million sites, 700,275 respond with a certificate on port 443; here are the top certificate authorities identified:

6b0zdvqx

  • Comodo: 192,646
  • GeoTrust Inc: 85,964
  • GoDaddy.com, Inc: 45,609
  • GlobalSign: 42,111
  • Let’s Encrypt: 38,190
  • Symantec Corporation: 27,612
  • DigiCert Inc: 27,588
  • cPanel, Inc: 24,195
  • thawte, Inc: 18,640
  • Starfield Technologies, Inc: 11,411

The first thing you may notice looking at that list, is that Comodo is the overwhelming leader – accounting for 28% of all certificates returned. Part of the reason for this is the partnerships they have: they are the certificate provider for CloudFlare, which offers TLS support to all customers, including those on their free plan, as well as various hosting providers. Here’s a breakdown:

  • CloudFlare: 73,339
  • HostGator: 9,499
  • BlueHost: 4,410
  • Others: 19,904

It’s quite exciting to see Let’s Encrypt not only in the top 10, but in the top 5 – for an organization that runs on less than $3M, they have done a remarkable job on impacting the industry.

For the two that started it all, StartCom has 10,221 and WoSign comes in at 3,965 – though in discussions with someone who has access to Google’s internal crawl logs, this is far from a complete picture of the usage of their certificates. As this scan only included the root (or www subdomain in case of an error) and port 443, this scan only captures some of the picture.

For invalid certificates (self-signed & otherwise untrusted), certificates with no issuer organization actually made it in the top 10, with 25,954 certificates; this is followed by localhost with 13,450 and Parallels Panel with 11,450 certificates.

The raw data for the scan is available here and was produced with a beta feature in YAWAST.

Thanks to Kenn White for the graphic!

Ruby + GCM Nonce Reuse: When your language sets you up to fail…

A couple hours ago, Mike Santillana posted to oss-security about a rather interesting find in Ruby’s OpenSSL library; in this case, the flaw is subtle – so much so that it’s unlikely that anyone would notice it, and it’s a matter of a seemingly insignificant choice that determines if your code is affected. When performing AES-GCM encryption, if you set the key first, then the IV, and you are fine – set the IV first, you’re in trouble.

Depending on the order you set properties, you can introduce a critical flaw into your application.

If you set the IV before the Key, it’ll use an empty (all zeros) nonce. We all (hopefully) know just how bad nonce reuse in GCM mode can be – in case you don’t recall, there’s a great paper on the topic. The short version is, if you reuse a nonce you are in serious trouble.

The Issue

Here’s the code that demonstrates the issue (based on code from Mike’s post, with some changes to better demonstrate the issue):

Each time this is called, a unique ciphertext should be produced thanks to the random IV (or nonce in this case), yet, that isn’t what happens:

In the first two cases, a random IV is used (cipher.random_iv), and should produce unique ciphertext every time it’s called, in the third case, we explicitly set a null IV – and if you notice, all three produce the same output. What’s happening is that the nonce is all zeros for all three cases – the random nonce isn’t being used at all, and thus the ciphertext is repeated each time the same message is encrypted. The fact that a null IV is used when no value is supplied is actually documented – it just so happens to be that setting an IV prior to setting the key is effectively the same as not setting one at all.

A bug 5 years in the making…

The cause of this issue is a test case that was found five years ago:

ruby -e 'require "openssl";OpenSSL::Cipher::AES128.new("ECB").update "testtesttesttest"'

What this code did was trigger a segmentation fault due to performing an update before the key was set. The workaround that was added was to initialize the Cipher with a null key to prevent the crash. At the time, this change wasn’t seen as being significant:

Processing data by Cipher#update without initializing key (meaningless usage of Cipher object since we don’t offer a way to export a key) could cause SEGV.

This ‘fix’ set the stage for this issue to come up. Setting a key that was meant to be overwritten caused a change in behavior in OpenSSL’s aes_gcm_init_key – instead of preserving a previously set IV, the IV is instead overwritten with the default null value. This isn’t exactly obvious behavior, and can only be seen by careful examination of the OpenSSL code.

So this is less a single bug, and more of a combination of odd behaviors that combine in a specific way to create a particularly nasty issue.

APIs That Lead To Failure

OpenSSL is notorious for its complicated API – calling it ‘developer unfriendly’ would be a massive understatement. I greatly appreciate the work that the OpenSSL developers do, don’t get me wrong – but seeing issues due to mistakes and misunderstandings of how OpenSSL works is quite common. Even in simple cases, it’s easy to make mistakes that lead to security issues.

It’s clear that this issue is just the latest in a long line caused by a complex and difficult to understand API, that makes what appears to be a simple change have far greater impact than anticipated. The result, is another API that you have to understand in great detail to use safely.

To make it clear, this isn’t the only case where order of operations can lead to failure.

The Workaround

The workaround is to just update code to move the call to cipher.random_iv to some point after the key is set – but this is something that shouldn’t matter. There are discussions going on now to determine how to correct the issue.

Ruby’s OpenSSL library is a core library that’s widely used, and performs security critical tasks in countless applications. For a flaw like this to go unnoticed is more than a little worrying. It’s vital that languages and libraries make mistakes as hard as possible – in this case, they failed.

Testing for SWEET32 with YAWAST

Testing for SWEET32 isn’t simple – when the vulnerability was announced, some argued that the best solution was to assume that if a TLS server supported any of the 3DES cipher suites, consider it vulnerable. The problem is, it’s not that simple. On my employer’s corporate blog, I wrote about practical advice for dealing with SWEET32 – and pointed out that there are ways around the vulnerability, and some are quite simple.

The easy options:

  • Disable the following cipher suites:
    • SSL_DH_DSS_WITH_3DES_EDE_CBC_SHA
    • SSL_DH_RSA_WITH_3DES_EDE_CBC_SHA
    • SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA
    • SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
    • TLS_RSA_WITH_3DES_EDE_CBC_SHA
    • TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
    • TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
    • TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
    • TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
  • Limit HTTP Keep-Alive to something reasonable (Apache & Nginx default to 100 requests)

If you don’t need to worry about users on old platforms, it’s easy – just disable the 3DES suites and you’re done. If you do need to worry about customers using Windows XP, disabling the 3DES cipher suites may not be an option, as the only other cipher suites they support use RC4. In that case, you need to limit Keep-Alive sessions.

In cases where 3DES suites are enabled, but reasonable / default limits are in place, SWEET32 isn’t an issue – as there’s no way to issue enough requests within the same session (and thus the same encryption key) to be able to exploit it.

Testing with YAWAST

To test for this, I added a new feature to YAWAST that will connect to the server using a 3DES suite, and issue up to 10,000 HEAD requests, to see how many requests the server will allow in a single session. If the server allows all 10,000 requests, there’s a good chance that the server is vulnerable to SWEET32. This method is used to determine that the server is likely vulnerable, without the massive data transfer and time required to actually verify that it’s vulnerable.

Testing for SWEET32 with YAWAST looks like this:

In this case I’m using YAWAST to run a ssl scan, using the --tdessessioncount parameter to instruct YAWAST to perform the SWEET32 test. In this case, you can see that the TLS session was ended after 100 requests (Connection terminated after 100 requests (TLS Reconnected)) – which is a clear indication that the server isn’t vulnerable. Had the server actually been vulnerable, this message would have been displayed:

[V] TLS Session Request Limit: Connection not terminated after 10,000 requests; possibly vulnerable to SWEET32

The use of 10,000 requests is somewhat arbitrary – it was selected as it’s far above the defaults used for the most common servers, and is small enough that it can be conducted in a reasonable time. It could be argued that it should be larger, to reduce false positives – it could also be argued that it’s excessive, as if it allows 5,000, it’s a reasonable bet that there’s no limit being enforced. This value may change over time based on community feedback, but for now, it’s a safe value that strikes a good balance.

Vulnerable or Possibly Vulnerable?

Unfortunately the only way to confirm that a server is truly vulnerable is to perform the attack, or at least a large part of it. There will be false positives using this technique, but that’s the cost of performing a quick test (that doesn’t involve transferring tens or hundreds of gigabytes of traffic).

When looking at the results, it’s important to understand that it indicates that the server is likely vulnerable, but it is not a confirmation of vulnerability.