Difficulty: Advanced

Module 8: Full Chain, Detection & Countermeasures

The gate opens, but the watchtower sees everything -- or does it?

The Complete Picture

This final module ties together the entire Hell's Gate chain, from initial PEB walk to syscall execution, and then examines the detection surface from the defender's perspective. Direct syscalls bypass userland hooks, but the kernel has its own instrumentation. Understanding these detection vectors is essential for both offensive research and defensive engineering.

The Complete Execution Chain

Let us trace the entire Hell's Gate flow from process startup to syscall completion:

Hell's Gate Complete Chain

1. Process starts; EDR DLL injected; ntdll stubs are hooked
2. Hell's Gate reads PEB → finds ntdll.dll base address
3. Parses ntdll PE headers → locates Export Address Table
4. Walks EAT, hashes each name, matches against target hashes
5. For each target: reads stub bytes, validates 4c 8b d1 b8 pattern
6. Extracts SSN from bytes [4:5] (or neighbor search if hooked)
7. Calls HellsGate(ssn) to set the SSN in global variable
8. Calls HellDescent(params...) → mov r10,rcx; mov eax,ssn; syscall
9. Kernel: KiSystemCall64 → SSDT dispatch → Nt* function executes
10. sysret → NTSTATUS returned to caller. EDR hooks never fired.

Detection Vector 1: Syscall Origin Validation

The most significant detection vector for direct syscalls is syscall origin validation. When the kernel receives a syscall, it can check where the syscall instruction was executed from (the return address in RCX). Legitimately, this should always be inside ntdll.dll's memory range.

C// Kernel-side syscall origin check (conceptual)
// In KiSystemCall64, RCX contains the return address (where syscall was called from)

PVOID syscallOrigin = (PVOID)TrapFrame->Rcx;  // Return address

// Get ntdll's memory range
PVOID ntdllBase = PsGetProcessNtdllBase(PsGetCurrentProcess());
SIZE_T ntdllSize = GetModuleSize(ntdllBase);

// Check if the syscall originated from within ntdll
if (syscallOrigin < ntdllBase ||
    syscallOrigin > (PBYTE)ntdllBase + ntdllSize) {
    // ALERT: syscall from outside ntdll!
    // This is a strong indicator of direct syscall usage
    LogSuspiciousSyscall(PsGetCurrentProcess(), syscallOrigin, SSN);
}

The Biggest Weakness of Direct Syscalls

When Hell's Gate executes syscall from its own ASM stub, the return address points to the Hell's Gate code (inside the .text section of the malicious binary), not into ntdll.dll. A kernel driver or ETW consumer can check this return address and immediately identify that the syscall did not originate from the legitimate ntdll stub. This is the primary detection mechanism modern EDRs use against direct syscall techniques.

Detection Vector 2: ETW Syscall Telemetry

Event Tracing for Windows (ETW) provides kernel-level telemetry that cannot be bypassed by userland techniques. Several ETW providers are relevant:

ETW ProviderWhat It CapturesDetection Capability
Microsoft-Windows-Threat-IntelligenceSensitive operations (memory allocation, thread creation in remote processes)Fires regardless of syscall origin; captures full parameters
Microsoft-Windows-Kernel-Audit-API-CallsSpecific security-relevant kernel API callsLogs syscall parameters and caller context
Microsoft-Windows-Kernel-ProcessProcess and thread lifecycle eventsThread creation detected even via direct syscall

The Threat Intelligence ETW provider (TI ETW) is particularly important. It is a kernel-level provider registered by the EtwTi (Threat Intelligence) routines in the kernel. It fires callback notifications for sensitive operations like cross-process memory writes, thread injection, and handle duplication -- regardless of whether the syscall came through ntdll or via a direct invocation.

C// Example: ETW TI provider detects NtAllocateVirtualMemory
// in a remote process, even via direct syscall
//
// The kernel function NtAllocateVirtualMemory internally calls:
//   EtwTiLogAllocExecVm()
// which generates an ETW event with:
//   - Caller process ID
//   - Target process ID
//   - Base address
//   - Size
//   - Protection flags
//   - Call stack (including the suspicious return address)
//
// The EDR's kernel-mode ETW consumer receives this event
// and can correlate it with other activity

Why ETW Is Hard to Evade

ETW instrumentation is embedded inside the kernel functions themselves. When NtAllocateVirtualMemory executes in the kernel, the ETW logging call happens inside that function's code path. There is no user-mode bypass for this -- the events fire regardless of how the syscall was invoked. Disabling or tampering with ETW providers from user mode (e.g., patching EtwEventWrite) is itself detectable and increasingly protected by Kernel Patch Protection (PatchGuard) and Hypervisor-protected Code Integrity (HVCI).

Detection Vector 3: Kernel Callbacks

Windows provides kernel callback mechanisms that EDR drivers register with to receive notifications about security-relevant events:

CallbackRegistration APIEvents
Process NotifyPsSetCreateProcessNotifyRoutineExProcess creation and termination
Thread NotifyPsSetCreateThreadNotifyRoutineThread creation (detects NtCreateThreadEx even via direct syscall)
Image Load NotifyPsSetLoadImageNotifyRoutineDLL/image loading (detects manual mapping attempts)
Object CallbacksObRegisterCallbacksHandle operations (open process, duplicate handle)
Minifilter CallbacksFltRegisterFilterFile I/O operations (payload drops, persistence)

These kernel callbacks fire based on the kernel operation, not on how the syscall was invoked. When NtCreateThreadEx creates a thread (whether called via ntdll or direct syscall), the thread creation notify callback fires. The EDR's kernel driver receives the notification with the process ID, thread ID, and start address -- enough to detect injection.

Detection Vector 4: Call Stack Analysis

Modern EDRs perform call stack inspection during kernel callbacks and ETW event processing. The call stack reveals the origin of the syscall:

C// Legitimate call stack for NtAllocateVirtualMemory:
//   ntdll!NtAllocateVirtualMemory+0x14      (syscall instruction)
//   KERNELBASE!VirtualAlloc+0x47             (wrapper)
//   myprogram!main+0x123                     (application code)
//   kernel32!BaseThreadInitThunk+0x14

// Suspicious call stack (direct syscall via Hell's Gate):
//   malware.exe!HellDescent+0x0A             (syscall from unknown module!)
//   malware.exe!main+0x456                   (caller)
//   kernel32!BaseThreadInitThunk+0x14

// The absence of ntdll in the call stack is a STRONG indicator

Indirect Syscalls: The Counter-Countermeasure

To evade syscall origin checks, some tools use indirect syscalls (e.g., RecycledGate, SysWhispers3). Instead of executing syscall from their own code, they set up registers and then JMP to the syscall instruction inside ntdll (at the known offset within the stub). This way, the return address points to ntdll, making the call appear legitimate. However, the call stack still reveals the unusual caller before ntdll.

The Evolving Arms Race

The Syscall Evasion Arms Race

Era 1: Attackers call Nt* functions normally → EDRs add userland hooks
Era 2: Attackers unhook ntdll → EDRs detect unhooking (integrity checks)
Era 3: Hell's Gate (direct syscalls) → EDRs add kernel callbacks + ETW
Era 4: Indirect syscalls (RecycledGate) → EDRs check call stack depth
Era 5: Stack spoofing + sleep obfuscation → EDRs add hardware breakpoints + ETW TI
Current: Multi-layer detection: hooks + ETW + callbacks + stack analysis + ML

Countermeasures Summary

Detection MethodWhat It CatchesBypass Difficulty
Userland hooks (ntdll)Normal API callsLow (Hell's Gate bypasses completely)
Syscall origin checksyscall from non-ntdll addressMedium (indirect syscalls evade)
ETW TI providerSensitive operations at kernel levelHigh (kernel-level, hard to tamper)
Kernel callbacksThread creation, handle ops, image loadsHigh (kernel-level, registered by driver)
Call stack analysisMissing ntdll frames, unusual callersMedium-High (stack spoofing is complex)
Thread start addressThreads starting in non-image-backed memoryMedium (ROP/thread hijacking can evade)
Memory scanningRWX pages, unbacked executable regionsMedium (W^X + encryption during sleep)

The Key Takeaway

Hell's Gate and direct syscalls are not a silver bullet. They effectively bypass one layer of defense (userland hooks), but modern EDR solutions deploy multiple overlapping detection layers. The technique remains valuable as part of a larger evasion strategy, but must be combined with stack spoofing, indirect syscalls, ETW tampering, and sleep obfuscation to evade the full detection stack. Understanding both the offensive technique and its detection surfaces makes you a better security researcher on either side.

OPSEC Considerations

If you are studying these techniques for red team operations or security research, consider the following operational security points:

C// OPSEC checklist for direct syscall implementations:

// 1. STATIC SIGNATURES
//    - DJB2 hash constants are known IOCs
//    - The HellsGate/HellDescent function names are signatured
//    - The wSystemCall global variable pattern is detectable
//    Solution: Use custom hash algorithms, randomize names, obfuscate

// 2. BEHAVIORAL INDICATORS
//    - PEB walking + EAT parsing sequence is suspicious
//    - Rapid succession of direct syscalls from non-ntdll code
//    - Allocate + Write + Protect + CreateThread pattern
//    Solution: Add delays, vary execution order, use alternative techniques

// 3. MEMORY INDICATORS
//    - The .text section contains syscall instructions (0F 05)
//    - The ASM stub pattern is identifiable in memory
//    - Shellcode in non-image-backed memory
//    Solution: Encrypt stubs when not in use, use stomped modules

// 4. BUILD ARTIFACTS
//    - Debug symbols, PDB paths, compiler metadata
//    - Import table reveals CRT usage (if not minimized)
//    - Section names and characteristics
//    Solution: Strip symbols, minimize imports, customize PE metadata

What Hell's Gate Started

The Hell's Gate paper and implementation by am0nsec and RtlMateusz catalyzed an entire field of research. Its core insight -- that SSNs can be resolved dynamically from ntdll stubs -- led to Halo's Gate, TartarusGate, SysWhispers2/3, RecycledGate, FreshyCalls, and dozens of other implementations. Understanding Hell's Gate gives you the foundation to understand every subsequent direct syscall technique, as they all build on the same principles of stub parsing and direct invocation.

Pop Quiz: Detection & Countermeasures

Q1: What is the primary way the kernel can detect that a syscall was executed via Hell's Gate instead of through ntdll?

When syscall executes, it stores the return address in RCX (which the kernel saves in the trap frame). If the syscall was executed from Hell's Gate's HellDescent function, this address points outside ntdll.dll. Kernel-side checks can compare this against ntdll's address range to detect direct syscalls.

Q2: Why can't direct syscalls evade the ETW Threat Intelligence provider?

ETW TI instrumentation is embedded within the kernel Nt* functions. When the kernel executes NtAllocateVirtualMemory (dispatched via SSDT regardless of caller), the ETW logging call inside that function fires an event. There is no user-mode bypass because the instrumentation is in kernel code.

Q3: What technique do indirect syscalls (e.g., RecycledGate) use to evade syscall origin checks?

Indirect syscalls set up all registers (including EAX with the SSN) and then JMP to the address of the syscall instruction within the actual ntdll stub. Since the syscall instruction executes from ntdll's address range, the return address stored in RCX points back into ntdll, passing origin validation checks.