As I’ve continued to analyze the round 2 results from the MITRE ATT&CK evaluation, I’ve had the opportunity to speak to more and more vendors that have indicated that one of the areas they struggled with was detecting the specific methods of PowerShell invocation used in the evaluation. Spoiler: It’s bad.

Consider the evolution of an offensive security practitioner from running someone else’s scripts to learning enough about the target environments to begin “living off the land” (LOL) and using native system capabilities to continue an attack once they’ve gotten a foothold. There are entire sites dedicated to “LOLBins” (Living Off The Land Binaries) or native programs that can be used for LOL, and PowerShell is probably the king of LOLBins.

What’s interesting is that MITRE deviated from using PowerShell Empire (a post-exploitation tool), as it did in round 1, to executing its own PowerShell scripts in this round . . . and the sudden drop in detections is a pretty good indication that these endpoint detection and response (EDR) products had been designed to look for post-exploitation tools instead of leveraging the Antimalware Scan Interface (AMSI) exposed by Microsoft for monitoring scripting within the environment, which provides the telemetry these vendors needed for detecting many of the events in this evaluation.

This Represents A Huge Vulnerability For Their Clients

While it’s true that adversaries only tend to work as hard as they have to, and many of them leverage these automated post-exploitation tools, you have to imagine they are using this evaluation to identify gaps and build a plan of attack after they’ve fingerprinted your security controls. So how bad is this? I pushed an update to my GitHub repo that generates a metric I’m calling “powerfail,” which counts the “misses” for each of these products in substeps where the documented procedure indicated that the attacker leveraged PowerShell. Vendors frequently highlight the number of “None” detections as “misses” where there wasn’t even telemetry recorded for an event. I think this is a pretty good place to use this metric because alerting on LOLBins would be extremely noisy, but you definitely want to ensure you have telemetry for it.

Let me reiterate: If your product is not generating telemetry detections for PowerShell, you’re blind. You may be able to detect an adversary coincidentally due to what the scripts are doing, but it makes it a lot harder to triage and investigate when you don’t have a record of what was being executed. Incidentally, many vendors do well detecting T1086: Powershell, but due to noise constraints, you can’t alert on this without knowing what’s being done with it. This problem is a lot bigger than T1086. Remember, THESE LOLBins ARE PART OF YOUR OPERATING SYSTEM AND ARE LIKELY IN FREQUENT LEGITIMATE USE IN YOUR ENVIRONMENT.

Make sure your vendor of choice doesn’t have a lot of “powerfails” and, if they do, that they have recognized this issue and remediated it.

Here’s another link to my GitHub repo.