Direct $MFT Parsing: Reading NTFS Below the Monitored API Layer

Original text: “Direct $MFT Parsing” — S12 — 0x12Dark Development, on Medium. This article summarises the technique in our own words for readers of core-jmp.org; the ASCII pipeline diagram and the short C++ excerpts below are reproduced with attribution to illustrate the discussion. For the complete C++17 header, main runner, and YARA rule, follow the source link.

Executive Summary

Almost every mainstream endpoint sensor, DFIR agent and telemetry pipeline sits on the same choke points: the Win32 file APIs (FindFirstFile, NtQueryDirectoryFile, GetFileAttributes) and the associated kernel callbacks. Because the sensor lives on the syscall path, anything that avoids that path is invisible to it. The Master File Table — the on-disk index that NTFS itself uses to describe every file, every stream and every deleted record on the volume — is exactly such a path. Reading the raw device and walking the $MFT gives an attacker or a forensic tool a full inventory of the filesystem without ever calling OpenFile on a single monitored object.

The technique is neither new nor exotic — it has been the working assumption of NTFS forensic parsers for years — but the source article by S12 is a compact, engineer-friendly recipe for doing it in modern C++, including a complete parser for the important attribute types ($STANDARD_INFORMATION, $FILE_NAME, $DATA, $OBJECT_ID, $REPARSE_POINT, $EA, $ATTRIBUTE_LIST, $SECURITY_DESCRIPTOR) and a matching YARA rule that lets defenders spot the pattern in a binary. This write-up walks through the shape of the technique and the surfaces defenders can actually watch.

Why direct $MFT parsing matters

Windows file APIs are convenient because they hide NTFS. That hiding is also what makes them monitorable. A userland FindFirstFile traverses directory indices through NtQueryDirectoryFile; a kernel minifilter attached at the filesystem stack will observe every one of those calls. Nothing about that path is a good hiding place for an attacker, and nothing about it is a reliable way for an incident responder to see files that have already been unlinked.

The Master File Table sits one level below. It is a special file ($MFT, record number 0) whose entries describe every file on the volume, live or deleted, with all of their timestamps, allocated data runs, alternate data streams and reparse-point metadata. Reading the volume as a block device and parsing that table produces:

every live file and directory, without going through FindFirstFile;
every deleted record that has not yet been overwritten — useful for forensics, dangerous when used offensively;
every alternate data stream (ADS) on every file, including streams that userland tools rarely enumerate;
the raw NTFS timestamps ($STANDARD_INFORMATION and $FILE_NAME), useful for timestomping detection and for reconstructing precise activity;
reparse points, extended attributes and security descriptors, all of which routine directory walks discard.

The pipeline

Conceptually the whole parser is a straight line from the raw volume handle to a resolved list of files. The source article summarises the flow as follows:

Raw Volume Handle
      ↓
FSCTL_GET_NTFS_VOLUME_DATA  →  MFT offset + record size
      ↓
Read Chunk of MFT Records
      ↓
For each record:
  └─ Validate Signature ("FILE")
  └─ Apply USN Fixup
  └─ Walk Attribute Chain
        └─ $STANDARD_INFORMATION → timestamps, flags
        └─ $FILE_NAME             → name, parent FRN
        └─ $DATA                  → size, data runs / resident data
      ↓
Build FRN Map
      ↓
Resolve Full Paths → Results[]

Pipeline diagram adapted from the original article.

How the technique works, step by step

1. Open the raw volume

The starting point is a handle to \.C: obtained with CreateFileW. Two details matter: the caller needs administrator rights (opening a volume as a block device is a privileged operation), and the flag FILE_FLAG_NO_BUFFERING must be set so that reads bypass the cache manager and land at sector-aligned offsets. From that handle a single DeviceIoControl call with FSCTL_GET_NTFS_VOLUME_DATA returns the layout the parser needs:

MftStartLcn — the logical cluster where $MFT begins;
BytesPerCluster — multiply by the above to get the absolute byte offset of the table;
BytesPerFileRecordSegment — the size of a single MFT record (1024 bytes on almost every modern install);
BytesPerSector — needed for read alignment and for USN fixup;
MftValidDataLength — divided by the record size, this is the number of records to walk.

2. Read the table in chunks

MFTs on modern installs are hundreds of megabytes to gigabytes; reading one record at a time is slow. The source article’s parser reads 128 records per chunk, aligned up to the sector size. The chunk buffer is allocated with VirtualAlloc (page-aligned by construction, which is what an unbuffered handle wants), and ReadFile is called from the current SetFilePointerEx position. Everything after that is in-memory record parsing.

3. Validate and apply USN fixup

Each MFT record begins with the four-byte signature FILE (0x454C4946 little-endian). Records that don’t match are either free slots or torn writes and must be skipped. Records that do match still need one repair step: NTFS uses the last two bytes of every sector inside a multi-sector record to store an update sequence number that lets the filesystem detect torn writes. The parser has to walk the update-sequence array, verify every sector tail matches the expected USN, and restore the original two bytes from the array before treating the record as trustworthy.

4. Walk the attribute chain

After fixup, everything of interest lives in the attribute list starting at FirstAttrOffset. Each attribute has a common header (TypeCode, RecordLength, resident/non-resident flag, name offset), followed by attribute-specific data. The parser advances through the chain by RecordLength until it hits the end marker 0xFFFFFFFF. The attributes the article treats in depth are:

$STANDARD_INFORMATION (0x10) — the classic four timestamps (creation, modification, MFT change, last access) plus the SI-level file attributes. The v3 tail (Windows 2000+) adds owner id, security id, quota charged and USN.
$FILE_NAME (0x30) — may appear multiple times per record, one per namespace (Win32, POSIX, DOS, Win32&DOS). The parser picks the highest-priority namespace and drops the legacy DOS name. This attribute also carries the parent File Reference Number (FRN), which is what makes full-path reconstruction possible.
$DATA (0x80) — the actual file content. If it is resident, the bytes live inside the record; if it is non-resident, the attribute stores a run list describing the on-disk extents. Multiple $DATA attributes with non-empty names are alternate data streams.
$OBJECT_ID (0x40) — per-object GUIDs used by the distributed link tracking service.
$REPARSE_POINT (0xC0) — symlinks, mount points and third-party filter tags; the parser decodes the two common reparse buffer shapes and returns substitute + print names.
$EA_INFORMATION (0xD0) and $EA (0xE0) — extended attributes (rarely used by user software, occasionally used by OS features).
$ATTRIBUTE_LIST (0x20) — the pointer-list used when a file’s attributes overflow into extension records.
$SECURITY_DESCRIPTOR (0x50) — per-file descriptors (increasingly rare; most SDs live centralised in $Secure).

5. Decode the data-run list

For non-resident $DATA attributes the on-disk location of a file is stored as a compact data-run list. Each run is one header byte encoding the width of a length field and the width of a signed LCN-delta field, followed by those two variable-width integers. The delta is signed (extents can go backwards on disk), and a zero-width LCN field marks a sparse run. Iterating the list yields VCN→LCN mappings that let the reader translate any logical file offset to a disk offset.

6. Rebuild the full path

MFT records only know their parent. To produce a path like C:UsersaliceDesktopnotes.txt the parser first collects every record’s best $FILE_NAME into a map keyed by FRN, then for each file walks up parent references until it hits the root (FRN 5), reversing the collected names as it goes. The FRN itself is a 64-bit value where the top 16 bits are a sequence number and only the low 48 bits identify the record — a mask (0x0000FFFFFFFFFFFF) is applied at every hop.

Reference implementation

The source article ships a single-header C++17 implementation (MftSnapshot) built around one static entry point:

std::vector<MftSnapshot::FileInfo> files =
    MftSnapshot::GetSnapshotViaMFT(L"\\.\C:", L"C:", /*includeDeleted=*/false);

Each FileInfo collects the parsed record header, an optional StandardInformation, a vector of FileNameAttr entries (one per namespace), a vector of DataStream objects (main $DATA plus every ADS, each with resident data or a decoded run list), plus optional ObjectIdentifier, ReparseData, EaInformation, ExtendedAttributes, AttributeListEntrys and SecurityDescriptorAttr. Derived fields (FullPath, IsDirectory, IsDeleted) are populated once the FRN map is complete.

Internally the implementation is a set of static helpers: ApplyFixup, ProcessRecord, per-attribute parsers (ParseSI, ParseFN, ParseData, ParseObjId, ParseReparse, ParseEaInfo, ParseEA, ParseAttrList, ParseSecDesc), the sign-extending ParseDataRuns, a BestFileName namespace-priority chooser and the BuildPath walker described above. The full source (~700 lines, packed struct definitions plus the class body) lives in the original article; readers who want to reuse it should copy from there.

Proof of concept

Running the reference binary against a live C: volume on the author’s test machine produces the following output. The 858 MB $MFT contains 879,104 records; after skipping free slots, extension records and (in this run) deleted entries, the parser resolves 191,086 live files:

#pragma once
#define NOMINMAX
#include <Windows.h>
#undef min
#undef max
#include <algorithm>
#include <iostream>
#include <unordered_map>
#include <unordered_set>
#include <vector>
#include <string>
#include <cstring>

#pragma pack(push, 1)

struct MFT_FILE_RECORD_HEADER {
    DWORD     Signature;
    WORD      UpdateSeqOffset;
    WORD      UpdateSeqCount;
    ULONGLONG Lsn;
    WORD      SequenceNumber;
    WORD      HardLinkCount;
    WORD      FirstAttrOffset;
    WORD      Flags;
    DWORD     BytesInUse;
    DWORD     BytesAllocated;
    ULONGLONG BaseFileRecord;  
    WORD      NextAttrInstance;
    WORD      Reserved;
    DWORD     RecordNumber;
};

struct NTFS_ATTR_HEADER {
    DWORD TypeCode;
    DWORD RecordLength;
    BYTE  FormCode;    
    BYTE  NameLength;  
    WORD  NameOffset;
    WORD  Flags;
    WORD  Instance;
};

struct NTFS_ATTR_RESIDENT {
    NTFS_ATTR_HEADER Common;
    DWORD ValueLength;
    WORD  ValueOffset;
    BYTE  IndexedFlag;
    BYTE  Reserved;
};

struct NTFS_ATTR_NON_RESIDENT {
    NTFS_ATTR_HEADER Common;
    ULONGLONG StartingVcn;
    ULONGLONG LastVcn;
    WORD      DataRunsOffset;     
    WORD      CompressionUnitSize;
    DWORD     Padding;
    ULONGLONG AllocatedSize;
    ULONGLONG FileSize;
    ULONGLONG InitializedSize;
    // ULONGLONG CompressedSize;   only present if CompressionUnitSize != 0
};

struct NTFS_FILE_NAME {
    ULONGLONG ParentFrn;
    ULONGLONG CreationTime;
    ULONGLONG LastWriteTime;
    ULONGLONG MftChangeTime;
    ULONGLONG LastAccessTime;
    ULONGLONG AllocatedSize;
    ULONGLONG EndOfFile;
    DWORD     FileAttributes;
    DWORD     ReparseOrEaInfo;
    BYTE      FileNameLength;      
    BYTE      FileNameNamespace;   // 0=POSIX 1=Win32 2=DOS 3=Win32&DOS
};

struct NTFS_STANDARD_INFORMATION {
    ULONGLONG CreationTime;
    ULONGLONG LastModificationTime;
    ULONGLONG MftModificationTime;
    ULONGLONG LastAccessTime;
    DWORD     FileAttributes;
    DWORD     MaximumVersions;
    DWORD     VersionNumber;
    DWORD     ClassId;
    // v3 extension, 72 bytes total (NTFS 3.0+, Windows 2000+)
    DWORD     OwnerId;
    DWORD     SecurityId;
    ULONGLONG QuotaCharged;
    ULONGLONG Usn;
};

struct NTFS_REPARSE_BUFFER_HEADER {
    DWORD ReparseTag;
    WORD  ReparseDataLength;
    WORD  Reserved;
    // reparse data follows
};

struct NTFS_SYMLINK_REPARSE_BUFFER {
    WORD  SubstituteNameOffset;
    WORD  SubstituteNameLength;
    WORD  PrintNameOffset;
    WORD  PrintNameLength;
    DWORD Flags;                   // 1 = relative symlink
    // WCHAR PathBuffer[] follows
};

struct NTFS_MOUNT_POINT_REPARSE_BUFFER {
    WORD SubstituteNameOffset;
    WORD SubstituteNameLength;
    WORD PrintNameOffset;
    WORD PrintNameLength;
    // WCHAR PathBuffer[] follows
};

struct NTFS_EA_INFORMATION {
    WORD  PackedEaSize;
    WORD  NeedEaCount;
    DWORD UnpackedEaSize;
};

struct FILE_FULL_EA_INFORMATION {
    DWORD NextEntryOffset;
    BYTE  Flags;
    BYTE  EaNameLength;           
    WORD  EaValueLength;
    // char  EaName[EaNameLength + 1]  
    // BYTE  EaValue[EaValueLength]    
};

struct NTFS_ATTR_LIST_ENTRY {
    DWORD     TypeCode;
    WORD      RecordLength;
    BYTE      AttributeNameLength; 
    BYTE      AttributeNameOffset; 
    ULONGLONG StartingVcn;
    ULONGLONG MftReference;        
    WORD      AttributeInstance;
    // WCHAR AttributeName[AttributeNameLength] follows at AttributeNameOffset
};

struct NTFS_SECURITY_DESCRIPTOR_HDR {
    BYTE  Revision;
    BYTE  Sbz1;
    WORD  Control;       // SE_DACL_PRESENT(0x0004), SE_SACL_PRESENT(0x0010), etc.
    DWORD OffsetOwner;   // byte offset from SD start; 0 = not present
    DWORD OffsetGroup;
    DWORD OffsetSacl;
    DWORD OffsetDacl;
};

static_assert(sizeof(NTFS_FILE_NAME) == 66, "pack check");
static_assert(sizeof(NTFS_STANDARD_INFORMATION) == 72, "pack check");
static_assert(sizeof(NTFS_ATTR_NON_RESIDENT) == 64, "pack check");
static_assert(sizeof(NTFS_ATTR_LIST_ENTRY) == 26, "pack check");
static_assert(sizeof(NTFS_SECURITY_DESCRIPTOR_HDR) == 20, "pack check");

#pragma pack(pop)





static constexpr DWORD  MFT_RECORD_SIGNATURE = 0x454C4946UL;
static constexpr DWORD  MFT_ATTR_STANDARD_INFO = 0x10UL;
static constexpr DWORD  MFT_ATTR_ATTRIBUTE_LIST = 0x20UL;
static constexpr DWORD  MFT_ATTR_FILE_NAME = 0x30UL;
static constexpr DWORD  MFT_ATTR_OBJECT_ID = 0x40UL;
static constexpr DWORD  MFT_ATTR_SECURITY_DESC = 0x50UL;
static constexpr DWORD  MFT_ATTR_DATA = 0x80UL;
static constexpr DWORD  MFT_ATTR_REPARSE_POINT = 0xC0UL;
static constexpr DWORD  MFT_ATTR_EA_INFORMATION = 0xD0UL;
static constexpr DWORD  MFT_ATTR_EA = 0xE0UL;
static constexpr DWORD  MFT_ATTR_END = 0xFFFFFFFFUL;

static constexpr WORD   MFT_FLAG_IN_USE = 0x0001;
static constexpr WORD   MFT_FLAG_DIRECTORY = 0x0002;

static constexpr BYTE   MFT_NS_POSIX = 0;
static constexpr BYTE   MFT_NS_WIN32 = 1;
static constexpr BYTE   MFT_NS_DOS = 2;
static constexpr BYTE   MFT_NS_WIN32_DOS = 3;


static constexpr DWORDLONG MFT_FRN_MASK = 0x0000FFFFFFFFFFFFULL;
static constexpr DWORDLONG MFT_ROOT_FRN = 5ULL;


struct MftEntry {
    std::wstring name;
    DWORDLONG    parentFrn;
    bool         isDirectory;
};

static inline DWORDLONG MftStripSeq(DWORDLONG frn) { return frn & MFT_FRN_MASK; }

// ─── MftSnapshot ─────────────────────────────────────────────────────────────

class MftSnapshot {
public:

    struct FileInfo {
        MFT_FILE_RECORD_HEADER RecordHeader;

        // $STANDARD_INFORMATION (0x10)
        struct StandardInformation {
            ULONGLONG CreationTime;
            ULONGLONG LastModificationTime;
            ULONGLONG MftModificationTime;
            ULONGLONG LastAccessTime;
            DWORD     FileAttributes;
            DWORD     MaximumVersions;
            DWORD     VersionNumber;
            DWORD     ClassId;
            DWORD     OwnerId;
            DWORD     SecurityId;
            ULONGLONG QuotaCharged;
            ULONGLONG Usn;
            bool      IsV3;
        };
        StandardInformation SI;
        bool HasSI = false;

        // $FILE_NAME (0x30) — one entry per namespace
        struct FileNameAttr {
            DWORDLONG    ParentFrn;
            ULONGLONG    CreationTime;
            ULONGLONG    LastWriteTime;
            ULONGLONG    MftChangeTime;
            ULONGLONG    LastAccessTime;
            ULONGLONG    AllocatedSize;
            ULONGLONG    EndOfFile;
            DWORD        FileAttributes;
            DWORD        ReparseOrEaInfo;
            BYTE         Namespace;
            std::wstring Name;
        };
        std::vector<FileNameAttr> FileNames;

        // $DATA (0x80) — one entry per stream (main + ADS)
        struct DataRun {
            LONGLONG Vcn;
            LONGLONG Lcn;    // -1 = sparse
            LONGLONG Length;
        };
        struct DataStream {
            std::wstring         Name;
            ULONGLONG            AllocatedSize;
            ULONGLONG            FileSize;
            ULONGLONG            InitializedSize;
            bool                 IsResident;
            std::vector<BYTE>    ResidentData;
            std::vector<DataRun> Runs;
        };
        std::vector<DataStream> DataStreams;

        // $OBJECT_ID (0x40)
        struct ObjectIdentifier {
            GUID ObjectId;
            GUID BirthVolumeId;
            GUID BirthObjectId;
            GUID DomainId;
        };
        ObjectIdentifier ObjId;
        bool HasObjId = false;

        // $REPARSE_POINT (0xC0)
        struct ReparseData {
            DWORD             Tag;
            std::wstring      SubstituteName;
            std::wstring      PrintName;
            std::vector<BYTE> RawBuffer;
        };
        ReparseData Reparse;
        bool HasReparse = false;

        // $EA_INFORMATION (0xD0)
        struct EaInformation {
            WORD  PackedEaSize;
            WORD  NeedEaCount;
            DWORD UnpackedEaSize;
        };
        EaInformation EaInfo;
        bool HasEaInfo = false;

        // $EA (0xE0)
        struct ExtendedAttribute {
            BYTE              Flags;
            std::string       Name;
            std::vector<BYTE> Value;
        };
        std::vector<ExtendedAttribute> EAs;

        // $ATTRIBUTE_LIST (0x20)
        struct AttributeListEntry {
            DWORD        TypeCode;
            DWORDLONG    MftReference;
            ULONGLONG    StartingVcn;
            std::wstring Name;
        };
        std::vector<AttributeListEntry> AttributeList;
        bool HasAttributeList = false;

        // $SECURITY_DESCRIPTOR (0x50)
        struct SecurityDescriptorAttr {
            WORD              Control;
            std::vector<BYTE> OwnerSid;
            std::vector<BYTE> GroupSid;
            std::vector<BYTE> Dacl;
            std::vector<BYTE> Sacl;
        };
        SecurityDescriptorAttr SecDesc;
        bool HasSecDesc = false;

        // Derived
        std::wstring FullPath;
        bool         IsDirectory = false;
        bool         IsDeleted = false;
    };


    // Returns one FileInfo per MFT base record.
    static std::vector<FileInfo> GetSnapshotViaMFT(const wchar_t* volumePath = L"\\\\.\\C:", const wchar_t* volumePrefix = L"C:", bool includeDeleted = false){
        std::vector<FileInfo> results;

        HANDLE hVol = CreateFileW(volumePath, GENERIC_READ,
            FILE_SHARE_READ | FILE_SHARE_WRITE, NULL,
            OPEN_EXISTING, FILE_FLAG_NO_BUFFERING, NULL);
        if (hVol == INVALID_HANDLE_VALUE) {
            std::wcerr << L"[-] CreateFileW: " << GetLastError() << L"\n";
            return results;
        }

        NTFS_VOLUME_DATA_BUFFER nvd = {};
        DWORD got = 0;
        if (!DeviceIoControl(hVol, FSCTL_GET_NTFS_VOLUME_DATA,
            NULL, 0, &nvd, sizeof(nvd), &got, NULL)) {
            std::cerr << "[-] FSCTL_GET_NTFS_VOLUME_DATA: " << GetLastError() << "\n";
            CloseHandle(hVol); return results;
        }

        const DWORD    recSize = nvd.BytesPerFileRecordSegment;
        const DWORD    sectSize = nvd.BytesPerSector;
        const LONGLONG mftOffset = nvd.MftStartLcn.QuadPart * nvd.BytesPerCluster;
        const DWORD64  totalRecs = static_cast<DWORD64>(nvd.MftValidDataLength.QuadPart / recSize);

        if (recSize == 0 || recSize > 65536 || (recSize & (recSize - 1)) != 0) {
            std::cerr << "[-] Bad BytesPerFileRecordSegment: " << recSize << "\n";
            CloseHandle(hVol); return results;
        }

        std::wcout
            << L"[*] MFT @ 0x" << std::hex << mftOffset << std::dec
            << L"  records: " << totalRecs
            << L"  (" << nvd.MftValidDataLength.QuadPart / 1024 / 1024 << L" MB)\n";

        LARGE_INTEGER li; li.QuadPart = mftOffset;
        if (!SetFilePointerEx(hVol, li, NULL, FILE_BEGIN)) {
            std::cerr << "[-] SetFilePointerEx: " << GetLastError() << "\n";
            CloseHandle(hVol); return results;
        }

        constexpr DWORD RECS_PER_CHUNK = 128;
        const     DWORD chunkSize = recSize * RECS_PER_CHUNK;
        BYTE* buf = static_cast<BYTE*>(
            VirtualAlloc(NULL, chunkSize, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE));
        if (!buf) {
            std::cerr << "[-] VirtualAlloc\n";
            CloseHandle(hVol); return results;
        }

        results.reserve(static_cast<size_t>(totalRecs / 3));
        std::unordered_map<DWORDLONG, MftEntry> frnMap;
        frnMap.reserve(static_cast<size_t>(totalRecs));

        DWORD64 recIdx = 0, nRead = 0;
        while (recIdx < totalRecs) {
            DWORD64 recsLeft = totalRecs - recIdx;
            DWORD   toRead = (recsLeft >= RECS_PER_CHUNK)
                ? chunkSize
                : static_cast<DWORD>(recsLeft * recSize);
            toRead = AlignUp(toRead, sectSize);

            DWORD bytesRead = 0;
            if (!ReadFile(hVol, buf, toRead, &bytesRead, NULL) || bytesRead == 0) break;

            DWORD recsInBuf = bytesRead / recSize;
            for (DWORD r = 0; r < recsInBuf; ++r) {
                if (recIdx + r >= totalRecs) break;

                FileInfo fi{};
                if (!ProcessRecord(buf + r * recSize, recSize, sectSize, includeDeleted, fi))
                    continue;

                // Build lightweight frnMap for FullPath resolution
                const FileInfo::FileNameAttr* best = BestFileName(fi);
                if (best) {
                    frnMap[fi.RecordHeader.RecordNumber] = MftEntry{
                        best->Name, best->ParentFrn, fi.IsDirectory
                    };
                }
                results.push_back(std::move(fi));
            }
            recIdx += recsInBuf;
            nRead += recsInBuf;

            if (nRead % 50'000 == 0 || recIdx >= totalRecs)
                std::cout << "\r[*] " << recIdx << " / " << totalRecs
                << "  live: " << results.size() << "   " << std::flush;
        }
        std::cout << "\n";

        VirtualFree(buf, 0, MEM_RELEASE);
        CloseHandle(hVol);

        for (auto& fi : results)
            fi.FullPath = BuildPath(frnMap, fi.RecordHeader.RecordNumber, volumePrefix);

        std::wcout << L"[+] Done: " << results.size() << L" entries\n";
        return results;
    }

private:
    static DWORD AlignUp(DWORD v, DWORD a) { return (v + a - 1) & ~(a - 1); }

    // Restore the two bytes that NTFS replaced with the USN at each sector tail.
    static bool ApplyFixup(BYTE* rec, DWORD recSize, DWORD sectSize) {
        auto* hdr = reinterpret_cast<MFT_FILE_RECORD_HEADER*>(rec);
        if (hdr->Signature != MFT_RECORD_SIGNATURE) return false;
        WORD* usa = reinterpret_cast<WORD*>(rec + hdr->UpdateSeqOffset);
        WORD  usn = usa[0];
        WORD  nSec = static_cast<WORD>(hdr->UpdateSeqCount - 1);
        for (WORD s = 0; s < nSec; ++s) {
            DWORD tail = static_cast<DWORD>(s + 1) * sectSize - sizeof(WORD);
            if (tail + sizeof(WORD) > recSize) break;
            WORD* slot = reinterpret_cast<WORD*>(rec + tail);
            if (*slot != usn) return false;
            *slot = usa[s + 1];
        }
        return true;
    }

    static const BYTE* ResidentVal(const BYTE* cur, const BYTE* attrEnd, DWORD& outLen) {
        const auto* hdr = reinterpret_cast<const NTFS_ATTR_HEADER*>(cur);
        if (hdr->FormCode != 0) return nullptr;
        const auto* res = reinterpret_cast<const NTFS_ATTR_RESIDENT*>(cur);
        const BYTE* val = cur + res->ValueOffset;
        if (val + res->ValueLength > attrEnd) return nullptr;
        outLen = res->ValueLength;
        return val;
    }

    static bool ProcessRecord(BYTE* rec, DWORD recSize, DWORD sectSize,
        bool includeDeleted, FileInfo& fi) {
        auto* hdr = reinterpret_cast<MFT_FILE_RECORD_HEADER*>(rec);

        // These fields are before any sector tail (offset < 510), safe to read before fixup
        if (hdr->Signature != MFT_RECORD_SIGNATURE) return false;
        fi.IsDeleted = !(hdr->Flags & MFT_FLAG_IN_USE);
        if (fi.IsDeleted && !includeDeleted) return false;
        if (hdr->BaseFileRecord != 0) return false; // extension record, no $FILE_NAME here

        if (!ApplyFixup(rec, recSize, sectSize)) return false;

        fi.RecordHeader = *hdr;
        fi.IsDirectory = (hdr->Flags & MFT_FLAG_DIRECTORY) != 0;

        if (hdr->FirstAttrOffset >= recSize) return false;
        const BYTE* cur = rec + hdr->FirstAttrOffset;
        const BYTE* end = rec + std::min(hdr->BytesInUse, recSize);

        while (cur + sizeof(NTFS_ATTR_HEADER) <= end) {
            const auto* attr = reinterpret_cast<const NTFS_ATTR_HEADER*>(cur);
            if (attr->TypeCode == MFT_ATTR_END)              break;
            if (attr->RecordLength < sizeof(NTFS_ATTR_HEADER)) break;
            if (cur + attr->RecordLength > end)                break;

            const BYTE* attrEnd = cur + attr->RecordLength;
            switch (attr->TypeCode) {
            case MFT_ATTR_STANDARD_INFO:  ParseSI(cur, attrEnd, fi);       break;
            case MFT_ATTR_FILE_NAME:      ParseFN(cur, attrEnd, fi);       break;
            case MFT_ATTR_DATA:           ParseData(cur, attrEnd, fi);     break;
            case MFT_ATTR_OBJECT_ID:      ParseObjId(cur, attrEnd, fi);    break;
            case MFT_ATTR_REPARSE_POINT:  ParseReparse(cur, attrEnd, fi);  break;
            case MFT_ATTR_EA_INFORMATION: ParseEaInfo(cur, attrEnd, fi);   break;
            case MFT_ATTR_EA:             ParseEA(cur, attrEnd, fi);       break;
            case MFT_ATTR_ATTRIBUTE_LIST: ParseAttrList(cur, attrEnd, fi); break;
            case MFT_ATTR_SECURITY_DESC:  ParseSecDesc(cur, attrEnd, fi);  break;
            }
            cur += attr->RecordLength;
        }

        return !fi.FileNames.empty(); // base records always have at least one $FILE_NAME
    }


    static void ParseSI(const BYTE* cur, const BYTE* attrEnd, FileInfo& fi) {
        DWORD len = 0;
        const BYTE* val = ResidentVal(cur, attrEnd, len);
        if (!val || len < 48) return;

        const auto* si = reinterpret_cast<const NTFS_STANDARD_INFORMATION*>(val);
        fi.SI.CreationTime = si->CreationTime;
        fi.SI.LastModificationTime = si->LastModificationTime;
        fi.SI.MftModificationTime = si->MftModificationTime;
        fi.SI.LastAccessTime = si->LastAccessTime;
        fi.SI.FileAttributes = si->FileAttributes;
        fi.SI.MaximumVersions = si->MaximumVersions;
        fi.SI.VersionNumber = si->VersionNumber;
        fi.SI.ClassId = si->ClassId;
        fi.SI.IsV3 = (len >= 72);
        if (fi.SI.IsV3) {
            fi.SI.OwnerId = si->OwnerId;
            fi.SI.SecurityId = si->SecurityId;
            fi.SI.QuotaCharged = si->QuotaCharged;
            fi.SI.Usn = si->Usn;
        }
        fi.HasSI = true;
    }

    static void ParseFN(const BYTE* cur, const BYTE* attrEnd, FileInfo& fi) {
        DWORD len = 0;
        const BYTE* val = ResidentVal(cur, attrEnd, len);
        if (!val || len < sizeof(NTFS_FILE_NAME)) return;

        const auto* fn = reinterpret_cast<const NTFS_FILE_NAME*>(val);
        const BYTE* fnEnd = val + sizeof(NTFS_FILE_NAME) + fn->FileNameLength * sizeof(WCHAR);
        if (fnEnd > val + len) return;
        if (NamespacePriority(fn->FileNameNamespace) < 0) return; // skip DOS

        FileInfo::FileNameAttr e;
        e.ParentFrn = fn->ParentFrn;
        e.CreationTime = fn->CreationTime;
        e.LastWriteTime = fn->LastWriteTime;
        e.MftChangeTime = fn->MftChangeTime;
        e.LastAccessTime = fn->LastAccessTime;
        e.AllocatedSize = fn->AllocatedSize;
        e.EndOfFile = fn->EndOfFile;
        e.FileAttributes = fn->FileAttributes;
        e.ReparseOrEaInfo = fn->ReparseOrEaInfo;
        e.Namespace = fn->FileNameNamespace;
        e.Name = std::wstring(reinterpret_cast<const WCHAR*>(fn + 1),
            fn->FileNameLength);
        fi.FileNames.push_back(std::move(e));
    }

    static void ParseData(const BYTE* cur, const BYTE* attrEnd, FileInfo& fi) {
        const auto* attr = reinterpret_cast<const NTFS_ATTR_HEADER*>(cur);

        FileInfo::DataStream s;

        // Stream name: empty = main $DATA, nonempty = ADS
        if (attr->NameLength > 0) {
            const BYTE* nameBytes = cur + attr->NameOffset;
            if (nameBytes + attr->NameLength * sizeof(WCHAR) <= attrEnd)
                s.Name = std::wstring(reinterpret_cast<const WCHAR*>(nameBytes),
                    attr->NameLength);
        }

        if (attr->FormCode == 0) {
            s.IsResident = true;
            DWORD len = 0;
            const BYTE* val = ResidentVal(cur, attrEnd, len);
            if (val) {
                s.AllocatedSize = s.FileSize = s.InitializedSize = len;
                s.ResidentData.assign(val, val + len);
            }
        }
        else {
            s.IsResident = false;
            if (cur + sizeof(NTFS_ATTR_NON_RESIDENT) <= attrEnd) {
                const auto* nr = reinterpret_cast<const NTFS_ATTR_NON_RESIDENT*>(cur);
                s.AllocatedSize = nr->AllocatedSize;
                s.FileSize = nr->FileSize;
                s.InitializedSize = nr->InitializedSize;
                const BYTE* runs = cur + nr->DataRunsOffset;
                if (runs < attrEnd)
                    s.Runs = ParseDataRuns(runs, static_cast<size_t>(attrEnd - runs));
            }
        }
        fi.DataStreams.push_back(std::move(s));
    }

    static void ParseObjId(const BYTE* cur, const BYTE* attrEnd, FileInfo& fi) {
        DWORD len = 0;
        const BYTE* val = ResidentVal(cur, attrEnd, len);
        if (!val || len < sizeof(GUID)) return;

        memcpy(&fi.ObjId.ObjectId, val, sizeof(GUID));
        if (len >= 4 * sizeof(GUID)) {
            memcpy(&fi.ObjId.BirthVolumeId, val + 16, sizeof(GUID));
            memcpy(&fi.ObjId.BirthObjectId, val + 32, sizeof(GUID));
            memcpy(&fi.ObjId.DomainId, val + 48, sizeof(GUID));
        }
        fi.HasObjId = true;
    }

    static void ParseReparse(const BYTE* cur, const BYTE* attrEnd, FileInfo& fi) {
        const auto* attr = reinterpret_cast<const NTFS_ATTR_HEADER*>(cur);
        if (attr->FormCode != 0) { fi.HasReparse = true; return; } // nonresident: tag unknown

        DWORD len = 0;
        const BYTE* val = ResidentVal(cur, attrEnd, len);
        if (!val || len < sizeof(NTFS_REPARSE_BUFFER_HEADER)) return;

        const auto* rbh = reinterpret_cast<const NTFS_REPARSE_BUFFER_HEADER*>(val);
        fi.Reparse.Tag = rbh->ReparseTag;

        const BYTE* data = val + sizeof(NTFS_REPARSE_BUFFER_HEADER);
        const BYTE* dataEnd = std::min(data + rbh->ReparseDataLength, val + len);
        fi.Reparse.RawBuffer.assign(data, dataEnd);

        auto readPath = [&](const BYTE* pathBuf, WORD off, WORD byteLen) -> std::wstring {
            if (pathBuf + off + byteLen > dataEnd) return {};
            return std::wstring(reinterpret_cast<const WCHAR*>(pathBuf + off),
                byteLen / sizeof(WCHAR));
            };

        if (rbh->ReparseTag == IO_REPARSE_TAG_SYMLINK &&
            data + sizeof(NTFS_SYMLINK_REPARSE_BUFFER) <= dataEnd) {
            const auto* sb = reinterpret_cast<const NTFS_SYMLINK_REPARSE_BUFFER*>(data);
            const BYTE* pathBuf = data + sizeof(NTFS_SYMLINK_REPARSE_BUFFER);
            fi.Reparse.SubstituteName = readPath(pathBuf, sb->SubstituteNameOffset, sb->SubstituteNameLength);
            fi.Reparse.PrintName = readPath(pathBuf, sb->PrintNameOffset, sb->PrintNameLength);
        }
        else if (rbh->ReparseTag == IO_REPARSE_TAG_MOUNT_POINT &&
            data + sizeof(NTFS_MOUNT_POINT_REPARSE_BUFFER) <= dataEnd) {
            const auto* mb = reinterpret_cast<const NTFS_MOUNT_POINT_REPARSE_BUFFER*>(data);
            const BYTE* pathBuf = data + sizeof(NTFS_MOUNT_POINT_REPARSE_BUFFER);
            fi.Reparse.SubstituteName = readPath(pathBuf, mb->SubstituteNameOffset, mb->SubstituteNameLength);
            fi.Reparse.PrintName = readPath(pathBuf, mb->PrintNameOffset, mb->PrintNameLength);
        }
        fi.HasReparse = true;
    }

    static void ParseEaInfo(const BYTE* cur, const BYTE* attrEnd, FileInfo& fi) {
        DWORD len = 0;
        const BYTE* val = ResidentVal(cur, attrEnd, len);
        if (!val || len < sizeof(NTFS_EA_INFORMATION)) return;

        const auto* ea = reinterpret_cast<const NTFS_EA_INFORMATION*>(val);
        fi.EaInfo.PackedEaSize = ea->PackedEaSize;
        fi.EaInfo.NeedEaCount = ea->NeedEaCount;
        fi.EaInfo.UnpackedEaSize = ea->UnpackedEaSize;
        fi.HasEaInfo = true;
    }

    static void ParseEA(const BYTE* cur, const BYTE* attrEnd, FileInfo& fi) {
        DWORD len = 0;
        const BYTE* val = ResidentVal(cur, attrEnd, len);
        if (!val) return; 

        const BYTE* ptr = val;
        const BYTE* valEnd = val + len;
        while (ptr + sizeof(FILE_FULL_EA_INFORMATION) <= valEnd) {
            const auto* ea = reinterpret_cast<const FILE_FULL_EA_INFORMATION*>(ptr);
            const BYTE* namePtr = ptr + sizeof(FILE_FULL_EA_INFORMATION);
            const BYTE* valuePtr = namePtr + ea->EaNameLength + 1;
            const BYTE* entryEnd = valuePtr + ea->EaValueLength;
            if (entryEnd <= valEnd) {
                FileInfo::ExtendedAttribute e;
                e.Flags = ea->Flags;
                e.Name = std::string(reinterpret_cast<const char*>(namePtr), ea->EaNameLength);
                e.Value.assign(valuePtr, entryEnd);
                fi.EAs.push_back(std::move(e));
            }
            if (ea->NextEntryOffset == 0) break;
            ptr += ea->NextEntryOffset;
            if (ptr >= valEnd) break;
        }
    }

    static void ParseAttrList(const BYTE* cur, const BYTE* attrEnd, FileInfo& fi) {
        fi.HasAttributeList = true;
        DWORD len = 0;
        const BYTE* val = ResidentVal(cur, attrEnd, len);
        if (!val) return;

        const BYTE* ptr = val;
        const BYTE* valEnd = val + len;
        while (ptr + sizeof(NTFS_ATTR_LIST_ENTRY) <= valEnd) {
            const auto* e = reinterpret_cast<const NTFS_ATTR_LIST_ENTRY*>(ptr);
            if (e->RecordLength < sizeof(NTFS_ATTR_LIST_ENTRY)) break;

            FileInfo::AttributeListEntry entry;
            entry.TypeCode = e->TypeCode;
            entry.MftReference = MftStripSeq(e->MftReference);
            entry.StartingVcn = e->StartingVcn;

            if (e->AttributeNameLength > 0) {
                BYTE off = e->AttributeNameOffset > 0
                    ? e->AttributeNameOffset
                    : static_cast<BYTE>(sizeof(NTFS_ATTR_LIST_ENTRY));
                const BYTE* nameEnd = ptr + off + e->AttributeNameLength * sizeof(WCHAR);
                if (nameEnd <= valEnd)
                    entry.Name = std::wstring(
                        reinterpret_cast<const WCHAR*>(ptr + off),
                        e->AttributeNameLength);
            }

            fi.AttributeList.push_back(std::move(entry));
            ptr += e->RecordLength;
        }
    }

    static void ParseSecDesc(const BYTE* cur, const BYTE* attrEnd, FileInfo& fi) {
        DWORD len = 0;
        const BYTE* val = ResidentVal(cur, attrEnd, len);
        if (!val || len < sizeof(NTFS_SECURITY_DESCRIPTOR_HDR)) return;

        const auto* sd = reinterpret_cast<const NTFS_SECURITY_DESCRIPTOR_HDR*>(val);
        fi.SecDesc.Control = sd->Control;

        // Copy a SID: total size = 8 + SubAuthorityCount * 4 (SubAuthorityCount is at byte[1])
        auto copySid = [&](DWORD offset, std::vector<BYTE>& dest) {
            if (offset == 0) return;
            const BYTE* p = val + offset;
            if (p + 8 > val + len) return;
            DWORD sidSize = 8u + p[1] * 4u;
            if (p + sidSize <= val + len) dest.assign(p, p + sidSize);
            };

        // Copy an ACL: total size stored as WORD at byte[2] of the ACL header
        auto copyAcl = [&](DWORD offset, std::vector<BYTE>& dest) {
            if (offset == 0) return;
            const BYTE* p = val + offset;
            if (p + 4 > val + len) return;
            WORD aclSize = *reinterpret_cast<const WORD*>(p + 2);
            if (p + aclSize <= val + len) dest.assign(p, p + aclSize);
            };

        copySid(sd->OffsetOwner, fi.SecDesc.OwnerSid);
        copySid(sd->OffsetGroup, fi.SecDesc.GroupSid);
        copyAcl(sd->OffsetSacl, fi.SecDesc.Sacl);
        copyAcl(sd->OffsetDacl, fi.SecDesc.Dacl);
        fi.HasSecDesc = true;
    }

    // Decode NTFS runlength encoded data run list
    static std::vector<FileInfo::DataRun> ParseDataRuns(const BYTE* ptr, size_t maxLen) {
        std::vector<FileInfo::DataRun> runs;
        const BYTE* end = ptr + maxLen;
        LONGLONG    curLcn = 0, curVcn = 0;

        while (ptr < end) {
            BYTE header = *ptr++;
            if (header == 0) break;
            int lenField = header & 0x0F;
            int lcnField = (header >> 4) & 0x0F;
            if (ptr + lenField + lcnField > end) break;

            LONGLONG runLen = 0;
            for (int i = 0; i < lenField; i++)
                runLen |= static_cast<LONGLONG>(*ptr++) << (i * 8);

            LONGLONG lcnDelta = 0;
            for (int i = 0; i < lcnField; i++)
                lcnDelta |= static_cast<LONGLONG>(*ptr++) << (i * 8);
            // Sign-extend the LCN delta
            if (lcnField > 0 && (lcnDelta >> (lcnField * 8 - 1)) & 1)
                lcnDelta |= ~((1LL << (lcnField * 8)) - 1);

            FileInfo::DataRun r;
            r.Vcn = curVcn;
            r.Lcn = (lcnField == 0) ? -1LL : curLcn + lcnDelta;
            r.Length = runLen;
            runs.push_back(r);

            if (lcnField > 0) curLcn += lcnDelta;
            curVcn += runLen;
        }
        return runs;
    }

    static const FileInfo::FileNameAttr* BestFileName(const FileInfo& fi) {
        const FileInfo::FileNameAttr* best = nullptr;
        int bestPrio = -1;
        for (const auto& fn : fi.FileNames) {
            int p = NamespacePriority(fn.Namespace);
            if (p > bestPrio) { bestPrio = p; best = &fn; }
        }
        return best;
    }

    static int NamespacePriority(BYTE ns) {
        switch (ns) {
        case MFT_NS_WIN32_DOS: return 3;
        case MFT_NS_WIN32:     return 2;
        case MFT_NS_POSIX:     return 1;
        case MFT_NS_DOS:       return -1;
        default:               return 0;
        }
    }

    static std::wstring BuildPath(const std::unordered_map<DWORDLONG, MftEntry>& map,
        DWORDLONG frn, const std::wstring& prefix) {
        std::vector<const std::wstring*> parts;
        DWORDLONG cur = MftStripSeq(frn);
        std::unordered_set<DWORDLONG> visited;

        while (cur != MFT_ROOT_FRN) {
            if (!visited.insert(cur).second) { parts.push_back(nullptr); break; }
            auto it = map.find(cur);
            if (it == map.end()) { parts.push_back(nullptr); break; }
            parts.push_back(&it->second.name);
            cur = MftStripSeq(it->second.parentFrn);
        }

        std::wstring path = prefix;
        for (int i = static_cast<int>(parts.size()) - 1; i >= 0; --i) {
            path += L'\\';
            path += parts[i] ? *parts[i] : L"<?>";
        }
        return path;
    }
};

int main(){
  std::vector<MftSnapshot::FileInfo> files = MftSnapshot::GetSnapshotViaMFT(L"\\\\.\\C:", L"C:", false);
  if (files.empty()) {
      std::cout << "Error getting files" << std::endl;
      return 1;
  }
  else {
      std::cout << "Files retrieved correctly" << std::endl;
      return 0;
  }
}

[*] MFT @ 0xc0000000  records: 879104  (858 MB)
[*] 879104 / 879104  live: 191086
[+] Done: 191086 entries
Files retrieved correctly

Sample output reproduced from the original article for illustration.

Detection

The article ships a YARA rule (DirectMFTParsing_RawNTFSEnumeration) tagged T1083 — File and Directory Discovery. Rather than trying to fingerprint the parser code itself, the rule looks for the small handful of primitives any implementation of this technique must contain in the compiled binary. The design is worth understanding even if you never load the rule as-is:

Volume-device paths as UTF-16 strings: \.C:, \.D:, …, \.PhysicalDrive, and the NT-native form ??C:. Any raw-block reader has to embed one of these.
NTFS-specific IOCTL codes as little-endian byte patterns: FSCTL_GET_NTFS_VOLUME_DATA (0x00090064) and FSCTL_GET_NTFS_FILE_RECORD (0x00090068).
The FRN mask 0x0000FFFFFFFFFFFF as an eight-byte constant — specific to file-reference-number handling and rarely appearing outside NTFS parsers.
The record signature "FILE" (0x454C4946), the attribute-chain end marker 0xFFFFFFFF, and the small set of common attribute type codes (0x10, 0x30, 0x80).
Debug/verbose strings such as MftStartLcn, BytesPerFileRecordSegment and $MFT — harmless in isolation but extremely diagnostic when combined with any of the IOCTLs above.

The rule combines those primitives into five variants of decreasing confidence, gated by a PE-signature check (uint16(0) == 0x5A4D) and a 20 MB filesize ceiling. Variant A (a volume path + FSCTL_GET_NTFS_VOLUME_DATA + the FRN mask) is the strongest signal — that three-way combination has essentially no reason to exist outside a direct-MFT reader — while Variant C (per-record FSCTL_GET_NTFS_FILE_RECORD instead of bulk read) is deliberately weaker and expected to match some legitimate disk-repair tools. The rule’s own note field flags this explicitly: expect matches on defragmenters and forensic suites, and combine with behavioural context before treating a hit as malicious. The complete rule — strings, variant conditions and metadata — is included in the original article.

Key Takeaways

Direct $MFT parsing gives a complete, path-resolved view of the filesystem — live entries, deleted entries, and alternate data streams — without calling a single monitored file API.
The whole pipeline is small: open the raw volume, one FSCTL_GET_NTFS_VOLUME_DATA for layout, chunked ReadFiles, USN fixup, attribute-chain walk, FRN-to-path resolution.
Detection is not about spotting the parser’s code — it’s about spotting the primitives it has to contain: volume paths, NTFS IOCTLs, the FRN mask, and NTFS field-name debug strings.
Any of those primitives in isolation is normal on Windows; the diagnostic combinations are (volume-path + FSCTL_GET_NTFS_VOLUME_DATA + FRN mask) or (record signature + FRN mask + attribute type codes + end marker).
Administrator (SeBackupPrivilege / raw-device access) is a hard prerequisite. If your threat model already assumes local admin, this technique isn’t an escalation; if it doesn’t, it’s an assumption that needs rechecking.

Defensive Recommendations

Watch the raw-volume open. Wire an ETW-Threat-Intelligence or minifilter callback on CreateFile targets matching \.?: or ???: from userland processes that are not on your allow-list (defragmenters, backup agents, forensic tools). Non-signed callers should never appear.
Alert on the NTFS IOCTLs. Instrument DeviceIoControl for control codes 0x00090064 (FSCTL_GET_NTFS_VOLUME_DATA) and 0x00090068 (FSCTL_GET_NTFS_FILE_RECORD). Almost no user-facing software issues these; the ones that do are known.
Deploy the YARA rule from the source article on your build/EDR content pipelines. Treat Variants A/B/D as high-confidence and route to triage; treat Variants C/E as low-confidence and correlate with process context.
Constrain who can open raw devices. Direct $MFT reads require administrator (technically the ability to obtain GENERIC_READ on a volume). Audit which service accounts and scheduled tasks have that privilege; anywhere it isn’t needed, remove it.
Add attribute-chain telemetry to your forensic pipeline. When you are the one parsing $MFT (IR, timeline reconstruction), compare $STANDARD_INFORMATION and $FILE_NAME timestamps for timestomping, list every $DATA stream (not only the main one) and always include deleted entries.
Assume ADS enumeration. If a compromised host runs a direct-MFT reader, the attacker sees every alternate data stream in one shot — do not rely on ADS obscurity for any secret material.
Post-incident, keep the volume for offline MFT capture. The same technique that hides the parser from EDR is exactly what a responder wants at rest: a raw $MFT extract preserves data the live filesystem no longer shows.

Conclusion

Direct $MFT parsing is a good example of a class of Windows techniques that only require administrator rights and a documented API surface to sidestep the sensors most organisations rely on. There is nothing to bypass at the kernel level, nothing to unhook and no exploit to develop — the visibility gap comes from the fact that on-disk NTFS structures live below the layer where monitoring is normally attached. That makes the technique valuable for red teams and for forensic responders in equal measure, and it makes the detection surface a matter of watching the small, distinctive set of primitives every implementation of it must contain.

Original text: “Direct $MFT Parsing” by S12 — 0x12Dark Development on Medium.