The Heartbleed Bug: How a Forgotten Bounds Check Broke the Internet

The Heartbleed bug is a critical buffer over-read flaw in several versions of the OpenSSL library that can reveal unencrypted information from the system memory of a server or client running a vulnerable version of OpenSSL. Because the library is used for secure communications over SSL/TLS and DTLS, attacks can reveal highly sensitive data, such as login credentials, TLS private keys, and personal information. The bug was fixed almost immediately after its disclosure in April 2014, but until hundreds of thousands of web servers that used the vulnerable library were patched, attackers could easily and undetectably extract sensitive information. Let’s take a closer look at one of the most serious and widespread security vulnerabilities in web history and see how just one buggy line of code could wreak havoc across the world.

How It All Began: Implementing the TLS Heartbeat Extension

For secure connections that use TLS (Transport Layer Security), a heartbeat message is a way of checking if a peer is still available without renegotiating the connection each time. The feature was proposed in 2012 with RFC 6520 and soon widely implemented in SSL/TLS libraries, including the open-source OpenSSL library. Unfortunately, the OpenSSL implementation of the TLS heartbeat extension submitted by Robin Seggelmann contained a small but fatal bug in the new feature. The first vulnerable version was OpenSSL 1.0.1, released in March 2012, with heartbeat support enabled by default. Ironically, this version was soon widely deployed on servers worldwide to increase security, as it added support for TLS 1.1 and 1.2 to eliminate vulnerabilities such as BEAST.

The Heartbleed Security Flaw in Detail

The TLS heartbeat protocol works a bit like an echo command. To check if the connection peer is still active, you send an arbitrary test message and if the connection is alive, the message should be echoed back immediately and exactly. The actual heartbeat message is an SSL3 record that includes not just the message payload but also its length (and padding, but this is not important for the flaw). To keep the connection alive, the test payload returned by the peer (typically a server) must have exactly the same content and length as in the heartbeat request. This is where OpenSSL software failed to implement one of the checks specified in RFC 6520, Section 4:

If the payload_length of a received HeartbeatMessage is too large, the received HeartbeatMessage MUST be discarded silently.

In the heartbeat extension as implemented in OpenSSL 1.0.1 through 1.0.1f, the server receives the heartbeat message and echoes back the requested payload with the requested length. However, the code does not check if the payload really is the same length as requested in the message. In other words, the server simply sends back a block of data from the start of the payload to the requested length. If the heartbeat message is well-formed, this is no problem, as the payload is always the same length as specified. But if the requested length exceeds the payload size, the server returns whatever is in memory after the local payload variable, thus leaking information. This is how the name Heartbleed came about – servers running vulnerable versions of OpenSSL could “bleed” sensitive data in heartbeat messages.

By sending a short heartbeat message with a large requested length, attackers could read up to 64 KB of raw memory content with each request. For servers running OpenSSL, this memory space could contain unencrypted user login credentials, session cookies, credit card numbers, and (critically) certificate private keys. By sending multiple malicious heartbeat requests, attackers could quickly amass vast amounts of valuable data, including secret keys that could allow unauthorized access to systems, identity theft, and man-in-the-middle attacks on ostensibly secure servers.

Reverse Heartbleed Vulnerability As with most discussion related to Heartbleed, this article mainly talks about attacks on servers, since these could affect thousands of users and organizations. However, the vulnerability can be exploited on any system running a vulnerable OpenSSL version, so a compromised server could use the same attack to read memory data from a vulnerable client application. This vulnerability was dubbed Reverse Heartbleed, and in 2014 it affected multiple computer applications and an estimated 50 million devices running Android 4.1.1.

Disclosure and Remediation

The bug was independently discovered by security researchers from Google and Finnish security firm Codenomicon (who coined the name “Heartbleed” and created a website explaining the bug). The vulnerability was secretly reported to the OpenSSL team and fixed in version 1.0.1g, released on April 7th, 2014. The fix itself was as trivial as the bug – just a matter of adding a value check and silently ignoring the heartbeat message if the declared and actual lengths differ. Heartbleed was added to the National Vulnerability Database as CVE-2014-0160, with the weakness classified as “Improper Restriction of Operations within the Bounds of a Memory Buffer (CWE-119)”.

Also on April 7th, 2014, news of the vulnerability was officially published. Website owners and server administrators were advised to immediately update to OpenSSL 1.0.1g. No runtime configuration parameter was available for the heartbeat extension, so it was not possible to disable just the buggy functionality without manually recompiling the OpenSSL library with the -DOPENSSL_NO_HEARTBEATS compiler option. All credentials, encryption keys, and SSL certificates used on the vulnerable servers might have been compromised, so the next recommendation was to reissue all security certificates for SSL/TLS from those servers, and to advise users to change their passwords. Finally, configuring the server to use key agreement protocols that ensure perfect forward secrecy could protect past communications from decryption even if current private keys were compromised, providing an extra layer of protection.

As of December 2019, a Shodan search for vuln:cve-2014-0160 still showed over 77,000 devices worldwide as vulnerable to Heartbleed, including over 18,000 Apache httpd servers.

Impact of the Heartbleed Bug

Security experts were unanimous that the vulnerability was extremely serious, as it undermined the very foundations of Internet security. SSL/TLS encryption and privacy could no longer be taken for granted, and anyone in the world could siphon sensitive data from trusted servers without leaving a trace. Because the vulnerability had existed since 2012 and there was no way of telling if or when it had been exploited, all private keys on vulnerable servers had to be considered compromised, requiring thousands of certificates to be revoked and reissued using new key pairs.

“Catastrophic” is the right word. On the scale of 1 to 10, this is an 11. – Bruce Schneier

The vulnerability affected a variety of software and firmware that relied on OpenSSL. These included not just open-source web servers like Apache and nginx but also computer applications and some network devices from Cisco, Juniper, and Apple. Many popular websites were also found to be vulnerable, including Yahoo! and gaming services such as Steam. Two in three web servers worldwide were vulnerable, and in the following days and weeks, millions of users were urged to change their passwords for multiple websites and services. Some data breaches were also reported. The global cost of dealing with the vulnerability was initially estimated at $500M.

Lessons from the Heartbleed Vulnerability

From a technical point of view, the vulnerability was a simple buffer over-read caused by a forgotten bounds check – a very common type of error that still tops the CWE Top 25 in 2019. The real issue was far more serious: how could such a trivial bug ever make it into security-critical software and then into millions of systems worldwide? The Heartbleed crisis highlighted the dangers of global reliance on open source software libraries that are often developed, tested, and maintained by small teams of volunteers with little or no financial backing. As Dan Kaminsky said, “We are building the most important technologies for the global economy on shockingly underfunded infrastructure”. In direct response to the financial and personal shortages of OpenSSL and other widely used open-source projects, the Linux Foundation created the Core Infrastructure Initiative (CII). Backed by technology industry leaders, the CII is intended to fund and support open-source projects that are critical to the functioning of global information systems such as the Internet. The crisis also prompted Google to form Project Zero – a full-time team of security researchers looking for zero-day vulnerabilities.

For website and web application owners and administrators, the takeaway from Heartbleed is that vulnerabilities can crop up in all code, including trusted and widely used libraries. Detecting flaws in external dependencies requires vulnerability scanning solutions that can look beyond the application code and find vulnerabilities such as Heartbleed before they are exploited by cybercriminals.