LibreSSL RNG bug fix: What's all the forking fuss about, ask devs

A bug found and fixed in LibreSSL, the OpenSSL fork maintained by OpenBSD developers, is “catastrophic" or "overblown", depending on whom you talk to.

Just days after the release of a portable version of the crypto library, a flaw was reported in LibreSSL's pseudo-random number generator – its PRNG, a vital component in strong encryption.‬

Andrew Ayer was able to write a program that could call LibreSSL's PRNG twice and get back the exact same stream of bits each time, which is not supposed to happen. Ayer, the founder of secure backup company Opsmate, described this bug as a “catastrophic failure of the PRNG.”

The OpenBSD project rejected this assessment as "overblown", and instead said the glitch was "minor". Nonetheless, the team promptly addressed the problem in LibreSSL version 2.0.2 and released it.

OpenBSD contributor Bob Beck reckoned a "contrived test program", such as the one produced by Ayer, was needed to reproduce the bug.

“While it’s a real issue, it’s actually a fairly minor one, because real applications don’t work the way the author describes, both because the PID (process identification number) issue would be very difficult to have become a real issue in real software, and nobody writes real software with OpenSSL the way the author has set this test up in the article,” Beck told Kaspersky Labs' Threatpost news service.

Ayer's fork_rand proof-of-concept code exploited the fact that fork()ing a program linked with LibreSSL produces a child process with the same PRNG state as the parent – meaning they will both spit out the same sequence of pseudo-random numbers.

This is not particularly brilliant because it means, for example, one process knows what another's random bit stream will look like, and these bit streams are used to generate secure encryption keys. Secrets will be leaked, in other words.

LibreSSL tries to detect when it is running in a fork()ed process by checking its process ID (PID): a change in PID means it is running in a new child, and duly resets its PRNG state to produce different numbers.

But the PID on Linux is 16 bits wide by default, and overflows when it hits that limit. If a program fork()s hard enough, a child could end up reusing a grandparent's PID and the aforementioned reseed check fails.

"You may think that fork_rand is a contrived example or that it's unlikely in practice for a process to end up with the same PID as its grandparent," wrote Ayer on his blog.

"You may be right, but for security-critical code this is not a strong enough guarantee. Attackers often find extremely creative ways to manufacture scenarios favorable for attacks, even when those scenarios are unlikely to occur under normal circumstances."

Bugs in PRNG components are a big deal because they weaken cryptographic systems. The NSA-championed Dual EC DRBG PRNG, which was offered as a default in the RSA BSAFE cryptography library, was found to be hopelessly flawed – so much so, it's believed Uncle Sam's spies deliberately knackered it with a backdoor.

LibreSSL forked from the OpenSSL cryptographic software library in April 2014 following the discovery of the password-leaking Heartbleed bug in OpenSSL; the vulnerability was severe enough that the OpenBSD gang decided to overhaul the crucial library, used in countless computers and devices for encryption and decryption.

Despite remaining critical of the way LibreSSL developers handled his bug report, Ayer remains broadly supportive of the project. "I really appreciate the work the LibreSSL devs are doing, especially their willingness to solicit feedback from the community and act on it," he said.

The fix, by the way, is to trap calls to fork() and act accordingly, rather than sense a PID change, but it may not be a perfect solution. ®