Adam Caudill

Security Leader, Researcher, Developer, Writer, & Photographer

Ruby + GCM Nonce Reuse: When your language sets you up to fail…

A couple hours ago, Mike Santillana posted to oss-security about a rather interesting find in Ruby’s OpenSSL library; in this case, the flaw is subtle – so much so that it’s unlikely that anyone would notice it, and it’s a matter of a seemingly insignificant choice that determines if your code is affected. When performing AES-GCM encryption, if you set the key first, then the IV, and you are fine – set the IV first, you’re in trouble.

Depending on the order you set properties, you can introduce a critical flaw into your application.

If you set the IV before the Key, it’ll use an empty (all zeros) nonce. We all (hopefully) know just how bad nonce reuse in GCM mode can be – in case you don’t recall, there’s a great paper on the topic. The short version is, if you reuse a nonce you are in serious trouble.

The Issue #

Here’s the code that demonstrates the issue (based on code from Mike’s post, with some changes to better demonstrate the issue):

#!/usr/bin/env ruby

require 'openssl'

def bin2hex(bin)
  bin.unpack('H*')
end

def encrypt(plaintext)
  cipher = OpenSSL::Cipher.new('aes-256-gcm')
  cipher.encrypt
  iv = cipher.random_iv # Notice here the IV is set before the key
  cipher.key = '11111111111111111111111111111111'
  cipher.auth_data = ""
  ciphertext = cipher.update(plaintext)
  ciphertext << cipher.final

  puts "[+] Encrypting: #{plaintext}"
  puts "[+] CipherMessage        (IV | Ciphertext): #{bin2hex(iv)} | #{bin2hex(ciphertext)}"
end

def encrypt_null_iv(plaintext)
  cipher = OpenSSL::Cipher.new('aes-256-gcm')
  cipher.encrypt
  cipher.iv = "\0"*16
  cipher.key = '11111111111111111111111111111111'
  cipher.auth_data = ""
  ciphertext = cipher.update(plaintext)
  ciphertext << cipher.final

  puts "[+] Encrypting: #{plaintext}"
  puts "[+] CipherMessage (No IV) (Ciphertext): #{bin2hex(ciphertext)}"
end

puts encrypt "This is some secret message."
puts encrypt "This is some secret message."
puts encrypt_null_iv "This is some secret message."

Each time this is called, a unique ciphertext should be produced thanks to the random IV (or nonce in this case), yet, that isn’t what happens:

[+] Encrypting: This is some secret message.
[+] CipherMessage        (IV | Ciphertext): ["b4eccdf06db4707822e7ad77"] | ["81092d16b62902d9985656253891dc800a5bb48fb1c4ad0b7bdf6054"]

[+] Encrypting: This is some secret message.
[+] CipherMessage        (IV | Ciphertext): ["4d0f19b972c7d90d7ee8b9ab"] | ["81092d16b62902d9985656253891dc800a5bb48fb1c4ad0b7bdf6054"]

[+] Encrypting: This is some secret message.
[+] CipherMessage (No IV) (Ciphertext):                                    ["81092d16b62902d9985656253891dc800a5bb48fb1c4ad0b7bdf6054"]

In the first two cases, a random IV is used (cipher.random_iv), and should produce unique ciphertext every time it’s called, in the third case, we explicitly set a null IV – and if you notice, all three produce the same output. What’s happening is that the nonce is all zeros for all three cases – the random nonce isn’t being used at all, and thus the ciphertext is repeated each time the same message is encrypted. The fact that a null IV is used when no value is supplied is actually documented – it just so happens to be that setting an IV prior to setting the key is effectively the same as not setting one at all.

A bug 5 years in the making… #

The cause of this issue is a test case that was found five years ago:

ruby -e 'require "openssl";OpenSSL::Cipher::AES128.new("ECB").update "testtesttesttest"'

What this code did was trigger a segmentation fault due to performing an update before the key was set. The workaround that was added was to initialize the Cipher with a null key to prevent the crash. At the time, this change wasn’t seen as being significant:

Processing data by Cipher#update without initializing key (meaningless usage of Cipher object since we don’t offer a way to export a key) could cause SEGV.

This ‘fix’ set the stage for this issue to come up. Setting a key that was meant to be overwritten caused a change in behavior in OpenSSL’s aes_gcm_init_key – instead of preserving a previously set IV, the IV is instead overwritten with the default null value. This isn’t exactly obvious behavior, and can only be seen by careful examination of the OpenSSL code.

So this is less a single bug, and more of a combination of odd behaviors that combine in a specific way to create a particularly nasty issue.

APIs That Lead To Failure #

OpenSSL is notorious for its complicated API – calling it ‘developer unfriendly’ would be a massive understatement. I greatly appreciate the work that the OpenSSL developers do, don’t get me wrong – but seeing issues due to mistakes and misunderstandings of how OpenSSL works is quite common. Even in simple cases, it’s easy to make mistakes that lead to security issues.

It’s clear that this issue is just the latest in a long line caused by a complex and difficult to understand API, that makes what appears to be a simple change have far greater impact than anticipated. The result, is another API that you have to understand in great detail to use safely.

To make it clear, this isn’t the only case where order of operations can lead to failure.

The Workaround #

The workaround is to just update code to move the call to cipher.random_iv to some point after the key is set – but this is something that shouldn’t matter. There are discussions going on now to determine how to correct the issue.

Ruby’s OpenSSL library is a core library that’s widely used, and performs security critical tasks in countless applications. For a flaw like this to go unnoticed is more than a little worrying. It’s vital that languages and libraries make mistakes as hard as possible – in this case, they failed.

Adam Caudill


Related Posts

  • Breaking the NemucodAES Ransomware

    The Nemucod ransomware has been around, in various incarnations, for some time. Recently a new variant started spreading via email claiming to be from UPS. This new version changed how files are encrypted, clearly in an attempt to fix its prior issue of being able to decrypt files without paying the ransom, and as this is a new version, no decryptor was available1. My friends at Savage Security contacted me to help save the data of one of their clients; I immediately began studying the cryptography related portions of the software, while the Savage Security team was busy looking at other portions.

  • Win by Building for Failure

    Systems fail; it doesn’t matter what the system is. Something will fail sooner or later. When you design a system, are you focused on the happy path, or are you building with the possibility of failure in mind? If you suffered a data breach tomorrow, what would the impact be? Does the system prevent loss by design, or does it just fall apart? Can you easily minimize loss and damage, or would an attacker have free rein once they get in?

  • Developers: Placing Trust in Strangers

    Much has been said, especially recently, about that mess of dependencies that modern applications have – and for those of us working in application security, there is good reason to be concerned about how these dependencies are being handled. While working on YAWAST, I was adding a new feature, and as a result, I needed a new dependency – ssllabs.rb. While most Ruby dependencies are delivered via Gems, ssllabs.rb is a little different – it pulls directly from Github:

  • PL/SQL Developer: Nonexistent Encryption

    (See here for another issue discovered during this research; Updates over HTTP & Command Execution.) PL/SQL Developer by Allround Automations has an option to store the user’s logon history with passwords – the passwords are encrypted with a proprietary algorithm. At this point, you should know how this is going to go. For those that don’t know, PL/SQL Developer is a tool for developers and database administrators to access Oracle – an essential tool in many enterprise environments.

  • Worried about the NSA? Try AES-512!

    …or, The Cost of Wild Speculation. “We need to boost our security – I think the NSA has broken everything we use. AES-256 is too weak, I don’t trust it. Find a way to implement AES-512.” Double-AES-256! It’d be easy, and double encrypting has never bitten us before. So, let’s write some code! def encrypt(msg, iv, key) return e(e(msg, iv, key.slice(0..31)), iv, key.slice(32..63)) end def decrypt(cipher, iv, key) return d(d(cipher, iv, key.