Module 2: Syscall Internals & SSN Resolution

How user-mode code crosses into Ring 0, and how to find the right number.

1. The x64 Syscall Instruction

The syscall instruction (opcode 0F 05) is the x64 mechanism for transitioning from user mode (Ring 3) to kernel mode (Ring 0). When it executes, the CPU performs these steps in hardware:

Once in kernel mode, KiSystemCall64 reads the value in RAX (the SSN) and uses it as an index into the System Service Descriptor Table (SSDT). The SSDT maps SSN 0x18 → NtAllocateVirtualMemory, SSN 0x50 → NtCreateFile, and so on.

2. The ntdll Syscall Stub

Every Nt* function in ntdll.dll is a small assembly stub. Here is the complete stub with hex bytes, exactly as it appears in memory:

x86-64 ASMNtAllocateVirtualMemory:
  4C 8B D1          mov r10, rcx        ; preserve 1st arg (RCX will be clobbered)
  B8 18 00 00 00    mov eax, 0x18       ; load SSN into EAX
  0F 05             syscall             ; Ring 3 → Ring 0
  C3                ret                 ; return to caller (12 bytes total)

Byte-by-Byte Breakdown

The SSN at offset +4 is what changes between different Nt* functions. NtAllocateVirtualMemory might be 0x18, NtProtectVirtualMemory might be 0x50, and these numbers change between Windows versions. This is why SSN resolution must be dynamic.

3. SSN Resolution Methods — History

The challenge: if EDR hooks overwrite the stub bytes, you can't just read the SSN from offset +4 anymore. Several techniques evolved to solve this:

4. MDSec Exception Directory Method

This is the method LayeredSyscall implements in its GetSsnByName() function. It exploits a structural property of ntdll: the Exception Directory (IMAGE_DIRECTORY_ENTRY_EXCEPTION) contains RUNTIME_FUNCTION entries that are ordered by function address, and syscall stubs are ordered by SSN.

Step-by-Step Walkthrough

C++// Walk PEB->Ldr->InLoadOrderModuleList
PPEB pPeb = (PPEB)__readgsqword(0x60);
PLDR_DATA_TABLE_ENTRY pEntry =
    (PLDR_DATA_TABLE_ENTRY)pPeb->Ldr->InLoadOrderModuleList.Flink;

// ntdll is typically the second entry (after the EXE itself)
pEntry = (PLDR_DATA_TABLE_ENTRY)pEntry->InLoadOrderLinks.Flink;
PVOID ntdllBase = pEntry->DllBase;

C++typedef struct _RUNTIME_FUNCTION {
    DWORD BeginAddress;    // RVA of function start
    DWORD EndAddress;      // RVA of function end
    DWORD UnwindData;      // RVA of unwind info
} RUNTIME_FUNCTION;

C++PIMAGE_DATA_DIRECTORY pExceptionDir =
    &pOptHeader->DataDirectory[IMAGE_DIRECTORY_ENTRY_EXCEPTION];

PRUNTIME_FUNCTION pRuntimeFunc = (PRUNTIME_FUNCTION)(
    (ULONG_PTR)ntdllBase + pExceptionDir->VirtualAddress);

DWORD numFunctions = pExceptionDir->Size / sizeof(RUNTIME_FUNCTION);

C++PIMAGE_DATA_DIRECTORY pExportDir =
    &pOptHeader->DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT];

PIMAGE_EXPORT_DIRECTORY pExports = (PIMAGE_EXPORT_DIRECTORY)(
    (ULONG_PTR)ntdllBase + pExportDir->VirtualAddress);

PDWORD pNames    = (PDWORD)((ULONG_PTR)ntdllBase + pExports->AddressOfNames);
PDWORD pFuncs    = (PDWORD)((ULONG_PTR)ntdllBase + pExports->AddressOfFunctions);
PWORD  pOrdinals = (PWORD)((ULONG_PTR)ntdllBase + pExports->AddressOfNameOrdinals);

C++// From LayeredSyscall's HookModule.cpp - GetSsnByName()
DWORD ssnCounter = 0;

for (DWORD i = 0; i < numFunctions; i++) {
    // Match RUNTIME_FUNCTION BeginAddress against export RVAs
    for (DWORD j = 0; j < pExports->NumberOfNames; j++) {
        if (pFuncs[pOrdinals[j]] == pRuntimeFunc[i].BeginAddress) {

            PCHAR funcName = (PCHAR)((ULONG_PTR)ntdllBase + pNames[j]);

            // Check if name starts with "Zw" (little-endian: 'wZ')
            if (*(USHORT*)funcName == 'wZ') {
                // Is this our target function?
                if (StringCompare(funcName + 2, targetName + 2)) {
                    return ssnCounter;  // SSN found!
                }
                ssnCounter++;
            }
            break;  // Move to next RUNTIME_FUNCTION
        }
    }
}

5. Why RUNTIME_FUNCTION Works

This method works because of two structural guarantees in ntdll.dll:

6. Additional Resolution: Syscall Address

Beyond the SSN, LayeredSyscall also needs the address of the syscall instruction within the stub. This is where hardware breakpoints will be set. The code scans forward from the function entry, looking for the two-byte sequence 0F 05:

C++// Scan for syscall (0F 05) within 25 bytes of function start
PVOID funcAddr = (PVOID)((ULONG_PTR)ntdllBase + funcRva);
for (DWORD offset = 0; offset < 25; offset++) {
    BYTE* p = (BYTE*)funcAddr + offset;
    if (p[0] == 0x0F && p[1] == 0x05) {
        // Found syscall instruction
        syscallAddr = (PVOID)p;
        retAddr     = (PVOID)(p + 2);  // ret is right after syscall
        break;
    }
}

This scan handles the case where an EDR hook might have modified early bytes but left the syscall/ret sequence intact (which is typical, since hooks redirect before the syscall is reached).

7. SSN Versioning & Why Dynamic Resolution Matters

SSNs are not stable across Windows versions. Microsoft can add, remove, or reorder system calls between updates. A hardcoded SSN table that works on Windows 10 21H2 may crash on Windows 11 23H2.

This is exactly why dynamic resolution methods like the Exception Directory approach are essential. They resolve the correct SSN at runtime by inspecting the actual ntdll loaded in the process, regardless of which Windows build is running.

C++// Initialization - resolve once, use many times
typedef struct _SYSCALL_INFO {
    DWORD  ssn;             // System Service Number
    PVOID  syscallAddr;     // Address of 0F 05 in the stub
    PVOID  retAddr;         // Address of C3 after syscall
} SYSCALL_INFO;

SYSCALL_INFO g_NtAllocateVM;
g_NtAllocateVM.ssn = GetSsnByName("ZwAllocateVirtualMemory");
// + find syscall/ret addresses...

Key Terminology