COFF Mixing – Hiding in Plain Sight

Executive Summary

Memory-injected payloads present distinct indicators of compromise: unbacked RX memory regions, empty call stacks, and process memory anomalies detectable by EDR systems. COFF mixing addresses this by merging capability code (typically PICO runners or DLL runners) with benign mathematical functions and linking the combined object file directly as a PE, eliminating the memory-injection step entirely. The resulting executable contains agent capabilities in image-backed memory with legitimate call stacks including kernel32 and ntdll, defeating memory-injection detection techniques while preserving full operational functionality.

This technique is particularly effective for replacing traditional shellcode runner payloads that inject agent code into their own memory space. By shifting the capability code into the PE binary itself during linking, defenders see only legitimate process behavior with proper call stacks and image-backed memory regions—exactly what Windows executables should look like.

The COFF Mixing Concept

COFF (Common Object File Format) mixing operates in two phases: first, the capability undergoes preparation including DFR (Domain Fronting Resolution) functionality and optimization passes; second, the processed capability merges into a PE-ready COFF using linker flags like +unwind to correctly establish x64 exception handling sections (.xdata and .pdata).

The workflow accepts capabilities written in PICO conventions (position-independent code runners) alongside benign compiled code, produces object files, and combines them via a linker spec. The final output is a standard Windows PE executable with no memory injection, no shellcode patterns, and no unbacked memory regions—all threat indicators are eliminated by design.

Call Stack Comparison

Memory-injected payloads exhibit distinctive call stacks with unbacked memory addresses:

# Child-SP          RetAddr               Call Site
00 0000000d`b7fffc58 00007ff4`dde6002d     USER32!MessageBoxA
01 0000000d`b7fffc60 0000000d`b7fffd50     0x00007ff4`dde6002d
02 0000000d`b7fffc68 00000133`170e1360     0x0000000d`b7fffd50
03 0000000d`b7fffc70 00000000`00000000     0x00000133`170e1360

Notice the unbacked addresses (00007ff4, 0000000d, 00000133)—these regions appear in debuggers with no backing image file, an immediate detection signal. By contrast, PE-linked COFF produces legitimate call stacks:

# Child-SP          RetAddr               Call Site
00 000000c3`5c1ff9c8 00007ff6`f9aa1932     USER32!MessageBoxA
01 000000c3`5c1ff9d0 00007ff6`f9aa15a9     out_x64+0x1932
02 000000c3`5c1ffa00 00007ff6`f9aa15a9     out_x64+0x15a9
03 000000c3`5c1ffa30 00007ff6`f9aa10d9     out_x64+0x10d9
04 000000c3`5c1ffad0 00007ff6`f9aa1436     out_x64+0x1436
05 000000c3`5c1ffb00 00007ffe`691de957     KERNEL32!BaseThreadInitThunk+0x17
06 000000c3`5c1ffb30 00000000`00000000     ntdll!RtlUserThreadStart+0x2c

Every address belongs to out_x64 (the PE image) or standard Windows libraries (KERNEL32, ntdll). The call stack shows proper program entry flow with image-backed memory throughout—precisely what legitimate applications display.

Symbols

Capabilities developed using PICO conventions expose function symbols visible in disassemblers like Binary Ninja. These symbols reveal intent: go, resolve_ext, ParseDLL, hash_function—names that signal a red team tool to any analyst. The strip command removes these symbols from COFF objects before linking, obfuscating the capability’s purpose within the final PE binary.

Before stripping:

After stripping:

Symbols are irrelevant once linked into PE because the linker resolves them to offsets. Keeping them post-link serves only to document intent for reverse engineers.

Function Ordering

The technique leverages function shuffling to bury capability code within benign functions. A linker option +disco randomly shuffles capability code interleaved with benign mathematical routines from libraries like bcpd (basic computational and probabilistic dynamics). The capability call can be embedded inside a function that performs legitimate computation, making static analysis significantly harder:

x64: bin
	$(CC_64) $(CFLAGS) -c src/main.c       -o bin/main.x64.o
	$(CC_64) $(CFLAGS) -c src/digamma.c    -o bin/digamma.x64.o
	$(CC_64) $(CFLAGS) -c src/dijkstra.c   -o bin/dijkstra.x64.o
	$(CC_64) $(CFLAGS) -c src/heap.c       -o bin/heap.x64.o
	$(CC_64) $(CFLAGS) -c src/kernel.c     -o bin/kernel.x64.o
	$(CC_64) $(CFLAGS) -c src/median.c     -o bin/median.x64.o
	$(CC_64) $(CFLAGS) -c src/misc.c       -o bin/misc.x64.o
	$(CC_64) $(CFLAGS) -c src/mt19937-64.c -o bin/mt19937-64.x64.o
	$(CC_64) $(CFLAGS) -c src/stat.c       -o bin/stat.x64.o
	$(CC_64) $(CFLAGS) -c src/util.c       -o bin/util.x64.o

The linker spec merges all objects together:

.capability
    make coff +unwind +disco

    load "bin/main.x64.o"
        merge
    load "bin/digamma.x64.o"
        merge
    load "bin/dijkstra.x64.o"
        merge
    load "bin/heap.x64.o"
        merge
    load "bin/kernel.x64.o"
        merge
    load "bin/median.x64.o"
        merge
    load "bin/misc.x64.o"
        merge
    load "bin/stat.x64.o"
        merge
    load "bin/util.x64.o"
        merge

With +disco enabled, the linker randomly interleaves functions from all objects. The capability’s go() function might execute from within a digamma() mathematical routine called indirectly through main(). Static analysis tools see only legitimate mathematical code and legitimate control flow, with no obvious red team intent.

Example integration point within benign code:

double digamma (double x)
{
  double r, f, t;

  r = 0;

  while (x<=5)
  { r -= 1/x;
    x += 1;
  }

  f = 1/(x*x);

  t = f*(-1/12.0 + f*(1/120.0 + f*(-1/252.0 + f*(1/240.0 + f*(-1/132.0
       + f*(691/32760.0 + f*(-1/12.0 + f*3617/8160.0)))))));

  go();
  return r + log(x) - 0.5/x + t;
}

The capability function go() executes during normal mathematical computation, entirely obscured by legitimate algorithmic logic.

Symbol Stripping Workflow

Stripping removes identifying function symbols from the final object before linking into the PE:

load "bin/main.x64.o"
    merge

strip "findFunctionByHash, resolve_ext, ParseDLL, go, hash_function, isForwardedFunction, GetDataDirectory, hash_module, findModuleByHash, resolve"

Once linked into the PE, symbol names are unnecessary (they’re resolved to memory offsets by the linker), so stripping them removes forensic artifacts without affecting execution.

Linker Spec Construction

A typical COFF mixing workflow uses a linker spec file to orchestrate the build:

capability.x64:
    push $OBJECT
        make coff +optimize +mutate

        load "../simple_pic/bin/services.x64.o"
            merge

        dfr "resolve" "ror13" "KERNEL32, NTDLL"
        dfr "resolve_ext" "strings"

        mergelib "../libtcg/libtcg.x64.zip"

    export

x64:
    .capability
        make coff +unwind

        load "bin/main.x64.o"
            merge

    export

The spec loads the capability (services.x64.o), applies DFR transformations (domain fronting resolution with string obfuscation), merges utility libraries, and exports. The main PE then loads the prepared capability, adds exception handling (+unwind), and produces the final executable.

Key Takeaways

• Eliminates memory-injection IOCs: Capability code lives in image-backed memory, producing legitimate call stacks with kernel32 and ntdll references.
• Bypasses EDR call stack analysis: Call stacks match expected Windows program flow, removing a primary detection vector.
• Symbol obfuscation: Stripping function symbols removes forensic intent signals while maintaining execution integrity.
• Function shuffling: The +disco flag randomly interleaves capability code with benign mathematical routines, obscuring control flow.
• Replaces shellcode runners: Shifts agent code from runtime-injected shellcode to static PE sections, fundamentally changing threat model.
• Preserves compatibility: Final executable is a standard Windows PE with no unusual characteristics, normal file size, and expected IAT entries.
• Build-time integration: Capability and benign code merge at the linker stage, producing a single self-contained binary with no external dependencies.

Defensive Recommendations

• Monitor benign library usage anomalies: Track unexpected symbols or function calls from common mathematical libraries (bcpd, gsl, etc.) in process execution.
• Analyze control flow graphs: Look for suspicious function calls during innocuous computation paths; use static analysis to detect capability markers.
• Harden linker environments: Restrict linker access in build pipelines; audit PE binaries for unusually large `.text` sections containing obfuscated function code.
• Behavioral monitoring at execution: Monitor for Windows API calls (DLL loading, memory allocation, networking) originating from unexpected code offsets within apparently legitimate functions.
• Baseline image-backed execution: Establish baselines for expected PE sections and IAT entries; flag binaries with unusual section sizes or imported functions for legitimate programs.
• Monitor for domain fronting techniques: Detect DFR transformations (string obfuscation, API hashing) that resolve module and function names at runtime; block suspicious domain-fronting patterns.
• Implement code integrity controls: Use signed PE verification; restrict execution of unsigned or self-modified binaries to prevent runtime capability injection.
• Enable ETW and Sysmon logging: Capture and correlate image load events, module load callbacks, and API call sequences to detect capability activation patterns.

Conclusion

COFF mixing represents a significant evolution in red team payload development by eliminating memory-injection detection vectors entirely. By merging capability code directly into PE binaries at link time and burying it within benign mathematical functions, operators defeat call stack analysis, memory region inspection, and symbol-based detection. The resulting executable is indistinguishable from legitimate software to many defensive tools, making COFF mixing particularly valuable for campaigns requiring long-term operational stability and evasion of modern EDR platforms.

Original text: “COFF Mixing – Hiding in Plain Sight” by Rasta Mouse at rastamouse.me.