By Chris Nyhuis, Vigilant CEO
Vigilant's threat intelligence team has been tracking and observing recent attacks taking advantage of CVE-2021-44228, a remote code execution (RCE) vulnerability in Apache Log4j v2 referred to as “Log4Shell." Apache_log4j v2 (Log4j) has been a vulnerability since 2013; however, it was recently made public and is being actively exploited by attackers at a high rate.
Log4j is a widely used module that allows for a Java-based application to better manage internal event logging. Logging systems can give attackers access to run code in your environments. Though the vulnerability is targeting a specific library, that library is in use by numerous popular applications and cloud services on varying operating systems.
2 key things to understand the RISK associated with this vulnerability:
1. This attack affects INTERNAL AND EXTERNAL SYSTEMS including cloud applications, virtual servers, switches, ticket systems, cloud providers etc. Simply locking down your external firewalls is not enough since attackers who are already inside organizations and internal threats can carry these attacks out to internal systems.
2. This vulnerability has existed since 2013*, which means that an attacker could have attacked you before this weekend. If you do not currently subscribe to MNDR from Vigilant or do not have the capability to do deep dive investigation, please reach to our team immediately. Just because you patched and/or do not see recent successful compromise doesn't mean that successful compromise didn't happen previous to you looking. This means that in addition to looking at this vulnerability you need to ensure that you do not have other command and control activity. We are seeing actors, as early as this morning, selling access to companies. *Note: a proof of concept for this was launched 9 months ago to show the depth and severity of this vulnerability.
Have we seen successful attack in client environments? As of today, Vigilant has seen over 15,000 attacks against organizations this weekend across all of our clients; however, only 1 company with successful execution which we have assisted in remediation.
What's the Latest?
This vulnerability allows an attacker to run unauthenticated remote code execution, through a specifically crafted string to a vulnerable component in Log4j v2 running on Apache. For continued technical and mitigation information about the vulnerability, please read the Vigilant Log4Shell Attack Resources available here.
- An attacker performs an HTTP request against a target system, which generates a log using Log4j v2.
- The vulnerability then causes the exploited process to reach out to the site and execute the payload.
- In many observed attacks, the attacker-owned parameter is a DNS logging system, intended to log a request to the site to fingerprint the vulnerable systems.
What is Vigilant Seeing?
The bulk of attacks that Vigilant has observed at this time have been related to mass scanning against DNS by attackers attempting to identify vulnerable systems; however, exploitation and post-exploitation activities have also been observed.
Based on the nature of the vulnerability, once the attacker has full access and control of an application, they can perform a broad aspect of objectives. Vigilant has observed activities including attacks like Cobalt Strike to enable credential theft, lateral movement, and exfiltrating data from companies.
This vulnerability goes back a long time and can affect systems all the way back to 2013.
The vulnerable versions observed at this point of Log4j v2 range from version 2.0 to 2.14.1. Additionally, there remains potential vulnerabilities in the deprecated 1.x versions.
What Immediate Actions Are We Recommending?
Due to the Critical nature of these vulnerabilities and the threat actor activity that we are seeing “in the wild,” Vigilant strongly recommends that you ensure the security of your data and environment by immediately updating any on-premises.
Before patching: Simply limit and cut off access to attackers.If you are not be able to patch quickly, or at all, a cheat code which we recommend first even prior to patching is to lock access down for affected systems down to specific IP's and or subnets in your environment. If you do that it helps cut off the ability to execute the attack and give you a moment to breathe and patch.
Ensure complete coverage of prevention - If you subscribe to our managed endpoint protection (MEDR) service, please make sure that the MEDR agent is deployed to your servers and running on any on-premises servers especially those running Apache. Please contact us when new systems have been enabled. If you are not sure if a vendor you use is compromised, or has patches, you can go to this link: https://gist.github.com/SwitHak/b66db3a06c2955a9cb71a8718970c592 If your vendor is not listed reach out to your Vendor directly. Implement the currently available patched version of Log4j that resolves this vulnerability, publicly available at https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-core/2.15.0/
Patch internal and external systems - this attack is not just against external facing systems. For applications executed through the Java Virtual Machine (JVM), this would take the form of a JVM argument of: Dlog4j2.formatMsgNoLookups=true Limit outbound access ports for systems in your environment specifically block: 1389 TCP, 39356 TCP (Blocking these is at your own risk, you need to verify that blocking these will not block any critical services in your environment.) Follow LDAP best practices by not allowing it inbound and outbound from your environment. Limit inbound Admin access to your firewalls to only internal IP's. Restrict direct access to your cloud infrastructures. If possible route traffic in and out of any cloud environment through a central detection/prevention choke point.
How is Vigilant Keeping Your Organization Secure from this attack?
Vigilant is continuously and actively hunting within our client environments for evidence of exposure to any threats – not just ones like this. Due to the nature of these attacks, how they are carried out, and multiple attack vectors, there is varying coverage based on the Vigilant services you subscribe to.
- If you have MNDR/CyberDNA®– CyberDNA® and our teams are constantly on the lookout for this kind of malicious activity so we can promptly investigate and notify our clients of additional actions to take to quickly mitigate/remediate the threat. As soon as this vulnerability was classified on Friday, we had detection for the associated scanning activity and DNS, HTTPS, and LDAP remote code execution attempts for the vulnerability. We are continuing to enhance our detection as new IOCs are identified in the wild and as this situation evolves.
- If you have MEDR-CrowdStrike- The best mitigation and detections for this vulnerability are primarily at the network layer, however, this is where defense-in-depth (aka a layered defense approach) comes into play. Currently, CrowdStrike has not publicly stated their coverage (or not) for this, however, they would catch the follow-on payload activity (or other remote actions) that an attacker would take to further compromise a host running any impacted products after the initial remote exploitation of the vulnerability. Make sure that Managed EDR is deployed to all publicly facing servers!
- If you only have MEDR- McAfee Level I – As of 12/11, McAfee is still reviewing their product coverage; however, the Vigilant team is looking for ways to add custom rules to McAfee in the interim based on known/new IOCs occurring in the wild. McAfee Level I does have full system inventory and does not allow us to do deeper analysis or remote remediation. If you would like us to scan and hunt on these systems, please reach out to your Vigilant CRS agent and request to be upgraded to MEDR McAfee Level II. We can easily upgrade your MEP level I systems within minutes.
- If you have MEDR- McAfee Level II or III – As of 12/11, McAfee is still reviewing their product coverage, however, the Vigilant team is looking for ways to add custom rules to McAfee in the interim based on known/new IOCs occurring in the wild. As McAfee Level II comes with full-on system hunting and file level inventory, our hunt and endpoint teams are actively hunting for this type of activity and tracking emerging threats related to these new vulnerabilities and exploits so we can promptly investigate and notify our clients of additional actions to take in order to quickly mitigate/remediate the threat.
What Are the Log4j2 Related Vulnerability Details?
CVE-2021-44228 Risk rating: Critical (10/10) Description:Apache Log4j2 2.14.1 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. From log4j 2.15.0, this behavior has been disabled by default. In previous releases (2.10) this behavior can be mitigated by setting system property "log4j2.formatMsgNoLookups" to “true” or by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class). Java 8u121 (see https://www.oracle.com/java/technologies/javase/8u121-relnotes.html) protects against remote code execution by defaulting "com.sun.jndi.rmi.object.trustURLCodebase" and "com.sun.jndi.cosnaming.object.trustURLCodebase" to "false".
As described above, the best mitigation and early detection for these kinds of vulnerabilities is at the network layer. If you are only taking advantage of one of our services, you potentially aren’t as protected as you could be and that is also why defense-in-depth is so important. With the right visibility into and across your network, as well as your endpoints, it is easier and faster to respond to these kinds of vendor vulnerabilities and detect if/when they are exploited by malicious actors.
If you would like to learn more about about subscribing to Vigilant's entire MDR suite of services in order to both strengthen your security posture and reduce your security risk, please contact your Vigilant Customer Relationship Specialist (CRS) via firstname.lastname@example.org
As always, thank you for being part of the Vigilant family and partnering with us to protect and defend your company!
Onward and upward,
Chris Nyhuis CEO Vigilant