HackCert
Intermediate 11 min read May 25, 2026

EDR Evasion Techniques: How Modern Malware Bypasses Security Systems

Explore the sophisticated EDR evasion techniques used by modern malware to bypass endpoint security systems and execute malicious payloads undetected.

Rokibul Islam
Red Team Operator
share
EDR Evasion Techniques: How Modern Malware Bypasses Security Systems
Overview

The deployment of Endpoint Detection and Response (EDR) solutions has fundamentally shifted the balance of power in cybersecurity defense. By moving away from static, signature-based Antivirus (AV) and adopting dynamic, behavioral monitoring, EDR platforms have made life exceptionally difficult for attackers. An EDR sensor acts as a surveillance camera inside the operating system, constantly recording process creations, memory injections, and network connections.

However, in the perpetual arms race of cybersecurity, adversaries—and the Red Teams that simulate them—have not remained stagnant. To execute their objectives without triggering the EDR's automated blocking mechanisms or alerting the Security Operations Center (SOC), attackers must employ highly sophisticated evasion techniques. Understanding these techniques is not just an exercise for malware developers; it is critical for detection engineers and security analysts who must continuously refine their EDR rules to catch the uncatchable. This guide will explore the intermediate and advanced strategies malware uses to slip past the watchful eyes of modern endpoint security.

The Mechanisms of EDR Surveillance

To evade an EDR, one must first understand how it observes the system. Most modern EDR solutions on the Windows operating system rely on a combination of two primary mechanisms to gain visibility:

  1. User-Mode API Hooking: The EDR injects its own Dynamic Link Library (DLL) into almost every process running on the system. This DLL "hooks" critical Windows API functions (like CreateProcess, VirtualAllocEx, or WriteProcessMemory). When a process attempts to call one of these functions, the hook redirects the call to the EDR's DLL first. The EDR inspects the arguments (e.g., "Is this process trying to inject code into another process?"), makes a determination, and then either blocks the action or passes it on to the legitimate Windows function.
  2. Kernel-Mode Callbacks (ETWti): Because user-mode hooks can be bypassed, EDRs also operate at the kernel level. They utilize built-in Windows features like Event Tracing for Windows Threat Intelligence (ETWti) and Kernel Callbacks. These mechanisms allow the EDR driver to receive highly privileged, un-hookable notifications directly from the Windows kernel whenever a process is created, a thread is spawned, or a handle is opened.

Successful evasion requires blinding the EDR at the user-mode level and obfuscating malicious intent to blend in with legitimate kernel telemetry.

Common EDR Evasion Techniques

Attackers utilize a variety of methods to circumvent these detection mechanisms, ranging from simple obfuscation to complex memory manipulation.

1. Unhooking (Bypassing User-Mode Hooks)

Since the EDR relies heavily on hooking the ntdll.dll library in user space, the most direct evasion technique is simply removing those hooks.

  • Direct Syscalls: Instead of calling the hooked API functions (like VirtualAlloc in kernel32.dll, which eventually calls NtAllocateVirtualMemory in ntdll.dll), malware can execute the system call (syscall) directly in assembly language. By bypassing the user-mode DLLs entirely and talking directly to the kernel, the malware evades the EDR's user-mode hooks completely.
  • Reflective DLL Loading / Manually Mapping: The malware can read a fresh, unhooked copy of ntdll.dll directly from the hard drive (C:\Windows\System32\ntdll.dll) and manually map it into its own memory space. It then resolves the addresses of the unhooked functions from this fresh copy and uses them instead of the hooked functions loaded by the operating system.

2. Living off the Land (LotL) and Proxy Execution

EDR solutions are highly tuned to detect unknown or suspicious executables performing malicious actions. To avoid this, attackers use "Living off the Land" binaries (LOLBins)—legitimate, Microsoft-signed executable files that are natively installed on Windows.

Instead of running a custom malicious executable, an attacker might use powershell.exe, wmic.exe, mshta.exe, or rundll32.exe to execute their payload. Because these are trusted system binaries, the EDR is less likely to block their initial execution. Furthermore, attackers use "Proxy Execution"—using one LOLBin to launch another, creating a complex process tree that confuses behavioral analysis engines.

3. Memory Evasion and Obfuscation

If an EDR scans the memory of a running process and finds the signature of a known tool (like Cobalt Strike or Mimikatz), it will immediately terminate the process. Therefore, the payload must remain hidden while in memory.

  • Sleep Obfuscation: Modern C2 frameworks utilize sleep obfuscation. When the malware beacon is not actively communicating with the attacker (which is most of the time), it encrypts its own executable memory region and puts the thread to sleep. To the EDR, the memory looks like random, benign data. When it is time to wake up, a small, unencrypted stub decrypts the memory, executes the task, and re-encrypts it before sleeping again.
  • Module Stomping: Instead of allocating new, highly suspicious memory (VirtualAllocEx with PAGE_EXECUTE_READWRITE permissions), the malware loads a legitimate, benign DLL that the process doesn't actually need. It then overwrites (stomps) the code section of that legitimate DLL with its malicious payload. The EDR sees a payload executing from memory backed by a legitimate file on disk, which appears far less suspicious.

4. Process Injection and Hollowing

To hide their actions and potentially escalate privileges, malware often injects itself into other, legitimate processes (like explorer.exe or svchost.exe).

  • Process Hollowing: The malware starts a legitimate process in a "suspended" state. It then "hollows out" the legitimate code from the process's memory and replaces it with the malicious payload. It then resumes the thread. The EDR and the Task Manager see a seemingly legitimate Microsoft process running, but it is actually executing the attacker's code. Evasion involves performing this injection using stealthy APIs (or direct syscalls) that the EDR might not be monitoring closely.

5. Blinding the EDR (ETW Patching)

While bypassing user-mode hooks is common, the kernel-level telemetry (ETW) is much harder to evade. However, ETW still relies on user-mode functions within ntdll.dll (specifically EtwEventWrite) to send some telemetry from the process back to the EDR.

Advanced malware will locate the EtwEventWrite function in memory and patch it. They overwrite the first few bytes of the function with a RET (return) instruction. Whenever the process tries to log an event to ETW, the function simply returns immediately without doing anything, effectively blinding the EDR to specific activities occurring within that process.

Defense and Mitigation: Catching the Evasion

Defending against EDR evasion requires a mature Security Operations Center (SOC) that does not rely solely on automated blocking.

  1. Kernel-Level Telemetry is King: Security teams must ensure their EDR solution heavily leverages kernel-mode drivers and ETWti. While user-mode hooks are easily bypassed via direct syscalls, the kernel always knows when memory is allocated or a process is created.
  2. Behavioral Heuristics over Signatures: Detection engineering must focus on the behavior of evasion techniques. For example, rather than trying to detect the payload itself, write rules to detect the act of unhooking (e.g., a process reading ntdll.dll from disk and mapping it), or detect the act of sleep obfuscation (e.g., a thread alternating between RW and RX memory permissions).
  3. Hunt for Anomalous LOLBin Usage: Implement strict application control and monitor the usage of LOLBins. Powershell.exe making an outbound network connection to an unknown IP, or rundll32.exe launching without command-line arguments, should trigger high-priority alerts regardless of the EDR's automated assessment.
  4. Memory Scanning: EDRs must be configured to periodically scan the memory of running processes, not just when a new process is created. This helps detect payloads that have been injected via module stomping or process hollowing after the initial execution phase.
Key Takeaways

The deployment of an EDR is a massive step forward in enterprise security, but it is not an impenetrable shield. Modern malware authors are highly skilled at dissecting EDR mechanics and developing techniques to unhook, obfuscate, and blind these sensors. By utilizing direct syscalls, memory encryption, and legitimate system binaries, attackers can successfully execute their objectives in the shadows of the operating system. For defenders, understanding these evasion techniques is paramount. It necessitates a shift from relying entirely on automated EDR blocking to proactive threat hunting, robust detection engineering, and a deep understanding of Windows internals to identify the subtle anomalies that reveal an attacker desperately trying to stay hidden.

Ready to test your knowledge? Take the EDR Evasion MCQ Quiz on HackCert today!

Related articles

back to all articles