KeyTrap: how to break a DNS server with a single packet

A group of researchers representing several German universities and institutes have discovered a vulnerability in DNSSEC, a set of extensions to the DNS protocol designed to improve its security, and primarily to counter DNS spoofing.

An attack they dubbed KeyTrap, which exploits the vulnerability, can disable a DNS server by sending it a single malicious data packet. Read on to find out more about this attack.

How KeyTrap works and what makes it dangerous

The DNSSEC vulnerability has only recently become public knowledge, but it was discovered back in December 2023 and registered as CVE-2023-50387. It was assigned a CVSS 3.1 score of 7.5, and a severity rating of “High”. Complete information about the vulnerability and the attack associated with it is yet to be published.

Here’s how KeyTrap works. The malicious actor sets up a nameserver that responds to requests from caching DNS servers – that is, those which serve client requests directly – with a malicious packet. Next, the attacker has the caching-server request a DNS record from their malicious nameserver. The record sent in response is a cryptographically-signed malicious one. The way the signature is crafted causes the attacked DNS server trying to verify it to run at full CPU capacity for a long period of time.

According to the researchers, a single such malicious packet can freeze the DNS server for anywhere from 170 seconds to 16 hours – depending on the software it runs on. The KeyTrap attack can not only deny access to web content to all clients using the targeted DNS server, but also disrupt various infrastructural services such as spam protection, digital certificate management (PKI), and secure cross-domain routing (RPKI).

The researchers refer to KeyTrap as “the worst attack on DNS ever discovered”. Interestingly enough, the flaws in the signature validation logic making KeyTrap possible were discovered in one of the earliest versions of the DNSSEC specification, published as far back as… 1999. In other words, the vulnerability is about to turn 25!

The origins of KeyTrap can be traced back to RFC-2035, the DNSSEC specification published in 1999

Fending off KeyTrap

The researchers have alerted all DNS server software developers and major public DNS providers. Updates and security advisories to fix CVE-2023-50387 are now available for PowerDNS, NLnet Labs Unbound, and Internet Systems Consortium BIND9. If you are an administrator of a DNS server, it’s high time to install the updates.

Bear in mind, though, that the DNSSEC logic issues that have made KeyTrap possible are fundamental in nature and not easily fixed. Patches released by DNS software developers can only go some way toward solving the problem, as the vulnerability is part of standard, rather than specific implementations. “If we launch [KeyTrap] against a patched resolver, we still get 100 percent CPU usage but it can still respond,” said one of the researchers.

Practical exploitation of the flaw remains a possibility, with the potential result being unpredictable resolver failures. In case this happens, corporate network administrators would do well to prepare a list of backup DNS servers in advance so they can switch as needed to keep the network functioning normally and let users browse the web resources they need unimpeded.