Executive Summary
Most LLM workflow discussion centres on prompt engineering and model selection, yet in practice the biggest gains in capability, cost and reliability come from the orchestration layer sitting above the model—what Andy Gill (ZephrFish) calls the harness. A harness controls the inputs, tools, prompts, models, state, validation gates and outputs for every stage of a pipeline. Without one, even the most powerful model burns tokens on redundant context, produces results that cannot be reproduced, and has no principled way to challenge its own findings.
This post surveys five publicly available harnesses oriented toward offensive security research—RAPTOR, Anthropic’s Code Reference Harness, baby-naptime, Evil Socket’s Audit, and Visa’s Vulnerability Agentic Harness—then covers the practical design decisions that matter when building your own: per-stage prompts, context budgeting, model routing, and RAG-backed memory for cross-run knowledge reuse. The author also releases harness-kit as a stripped-back template implementing a five-stage recon→hunt→validate→trace→report pipeline.
What is a Harness?
In the context of AI-assisted workflows, a harness is the orchestration layer that wraps around an LLM. It defines what inputs each stage receives, which tools are available, which prompts govern the model’s behaviour, what constitutes a valid output, and how results flow to the next stage. MCPs—Model Context Protocol servers that expose callable functions such as “run a shell command” or “decompile a binary”—live inside that layer. They supply the tool inventory, but the harness decides when each tool is invoked, with what context, and how the result is consumed.
For offensive security research the harness must answer a sequence of questions at each stage: what data should be collected, which tools should run and why, which model is best suited for this particular task, how much context is actually required, and how prior findings can be reused rather than rediscovered. It should also enforce a clear hand-off point where the model stops reasoning and returns control to the operator. More capable harnesses go further, dynamically selecting between models, invoking external APIs, and routing work between specialised agents—ensuring that each agent receives only the context its stage actually needs, which keeps token burn down and reasoning consistent across runs.
Trying to coerce useful work out of LLMs without the harness and verification layer is like herding cats—scale that up to multiple parallel agents and it’s closer to supervising a room full of drunk toddlers, each convinced they’re helping, none of them checking with each other and falling over the next.
Andy Gill, ZephrSec
For anyone wanting to track token consumption across a Claude Max subscription versus equivalent API spend, the author released TokenBurn for exactly that purpose.
Before We Begin: A Map of Public Harnesses
Several publicly available harnesses are worth studying before building your own. All of the following except Google’s (never released) Nap Time are open-source and actively maintained. RAPTOR is the one the author uses most heavily, but each has a distinct design philosophy that makes it useful as either a starting point or a complement to other tools.
Harnesses in Practice: RAPTOR
RAPTOR builds a structured vulnerability research pipeline around Claude Code. Rather than issuing a single prompt and waiting for findings, it orchestrates static analysis, binary analysis, fuzzing, vulnerability validation and exploit generation into a coherent multi-stage workflow.
The design separates concerns across two layers. A Python execution layer runs the tools and can be driven from CI to produce structured SARIF output entirely independently of the AI reasoning component. The Claude Code decision layer sits above it, determining what to run and how to interpret the results. This separation means the orchestration logic can be tested and iterated on without touching the model, which is a significant practical advantage when refining a research pipeline.
The validation pipeline runs across six stages. Stages A through D assess whether a vulnerability pattern is genuine, map the attacker reach required to trigger it, perform a line-by-line code review against the finding, and produce a final ruling with CVSS scoring. Stage E evaluates binary-level feasibility—checking ASLR and RELRO configurations, gadget availability, and applying Z3 SMT constraint solving to assess one-gadget applicability. Stage F runs a final contradiction check before any finding is promoted. The author uses RAPTOR as a Git submodule, primarily for the static-analysis stages and newer Frida-based dynamic exploration of Windows applications.
Harnesses in Practice: Anthropic Code Reference Harness
Anthropic’s reference harness is aimed specifically at C/C++ targets where execution-verified findings are the goal. It runs an autonomous find-grade-patch pipeline inside AddressSanitizer (ASAN) instrumented Docker containers, and every finding is accompanied by a binary proof-of-concept that reproduces the crash against the instrumented build—removing ambiguity about whether an issue is genuinely reachable.
The workflow runs in four phases. vulnpipeline_recon maps the attack surface and identifies investigation focus areas. vulnpipeline_run launches independent fuzzing agents against the ASAN build, collecting crash PoCs when they occur. vulnpipeline_report grades each unique crash as passed, borderline, DoS-only or low-impact. vulnpipeline_patch produces a source-level fix, rebuilds the target and re-runs the original PoC to confirm resolution. The harness is deliberately scoped to C/C++ projects that supply a Dockerfile, a build script and an instrumentable ASAN configuration—but for targets meeting those requirements it offers a strong complement to purely static analysis approaches.
Harnesses in Practice: Project Zero Nap Time & Baby Naptime
Google’s Project Nap Time was never publicly released, but an open-source equivalent exists: baby-naptime. It implements a single-agent runtime exploitation loop for C/C++ binaries. The model works against a live running binary in a tight feedback cycle: propose an approach, execute it, observe the output, revise the hypothesis, and repeat. Running this loop across dozens of iterations with real runtime signals produces a qualitatively different result from prompting a model to reason about a binary from static context alone—it gives the model the same ground-truth feedback a human reverser has during a live session.
Harnesses in Practice: Evil Socket’s Audit Framework
Audit by evilsocket takes a more language-agnostic approach. It does not require a clean build system, Docker setup or runtime instrumentation, making it applicable to repositories where those prerequisites are absent. The framework runs an eight-stage Claude Code pipeline that maps the codebase, identifies trust boundaries, reviews prior security fixes, and launches parallel investigation agents against focused workstreams. Findings are validated and deduplicated, then passed through a trace stage that must demonstrate attacker-controlled input reaching a vulnerable sink before anything is included in the final report.
That trace gate provides more discipline than a naive multi-agent code review, even without the runtime certainty of a reproduced ASAN crash. In the author’s experience the framework required meaningful tuning before producing consistently useful output, and the quality of results depends heavily on how well the recon tasks, prompts, model selection and codebase partitioning are configured. Large or unusual codebases can produce duplicated effort, shallow reviews or wasted cycles on low-value areas. It is better treated as a configurable research pipeline than a ready-to-run tool.
Harnesses in Practice: Visa’s Vulnerability Agentic Harness (VVAH)
VVAH sits closest to Audit in design but places greater emphasis on threat modelling and taint-flow analysis before agents begin hunting. The pipeline inventories the repository, maps trust boundaries, assigns specialist review lenses to each workstream, runs an adversarial second pass against initial findings, and produces both SARIF and Markdown reports. Importantly, it treats all results as triage candidates rather than confirmed vulnerabilities.
This makes it well suited for broad coverage across languages and repositories without reliable builds, but it carries the standard limitations of LLM-led source pipelines: results still require human review and tuning. The call graph is seeded by an LLM and reinforced with regex rather than derived from a full AST, so dynamic dispatch, reflection and framework-level routing can be missed. Unlike Anthropic’s harness it does not prove exploitability through runtime execution, and unlike RAPTOR it does not lean as heavily on external analysis tools and SMT solvers for binary-level validation.
Designing and Building Your Own Harness
One of the most common mistakes when building a harness is using a single system prompt for the entire pipeline. Each stage serves a different purpose and needs a prompt designed specifically for that purpose. An agent mapping a codebase needs different framing from one developing exploit hypotheses, and both need different framing from a verification agent whose job is to find reasons a finding is wrong. The mapping stage might return structured JSON covering file paths, entry points and dependencies; a later stage uses that as context for attack surface reasoning; the verification stage should be actively instructed to look for contradictions.
A good example of this pattern is the revalidate skill in Scrutineer. When the security-deep-dive skill produces a High or Critical finding, revalidate checks it against the repository’s git history and returns one of four verdicts: true_positive, false_positive, already_fixed or uncertain. Only findings marked true_positive advance to the verify stage, where the code is tested against the current HEAD. This gate keeps expensive validation work focused on findings most likely to be genuine.
Context Windows and Budgets
Treat the context window as a budget and manage it explicitly from the start. A frequent failure mode in early harnesses is passing raw files, full scanner output and entire conversation histories into every stage—more context is not automatically better when most of it is irrelevant to the question currently being asked.
Effective context management means: retrieving only the code paths relevant to the current hypothesis, summarising noisy tool output before it enters a prompt, maintaining a short rolling summary of completed work rather than the full history, and discarding resolved tasks once their results are stored elsewhere. As a rough guide, a single-function analysis often fits within around 8K tokens, while synthesising findings across multiple areas may need closer to 32K. Fuzzer output and scanner logs should typically be reduced to a few hundred useful tokens before entering any prompt. Design the context strategy before building the pipeline, because retrofitting it later is considerably more painful.
The Orchestration Layer
In the author’s own setup the orchestration layer sits above eight MCP servers and is composed of Claude Code skills organised into experts and workers, connected with bespoke glue code. The MCPs provide the tools; the orchestration layer decides which tools to call, in what order, and how to handle each result.
The publicly released harness-kit is a deliberately constrained template of this approach. Its workflow is five stages: recon → hunt → validate → trace → report. Recon maps the target. Hunt investigates focused hypotheses against it. Validate looks for reasons each finding is wrong. Trace proves whether attacker-controlled input actually reaches the vulnerable sink. Only findings that pass all gates reach the reporting stage.
Stages exchange structured artefacts rather than sharing one long conversation, which makes the pipeline easier to inspect, rerun and modify independently. Hunt workers can run narrow tasks in parallel with defined context budgets. Model routing is built into the design: cheaper, faster models handle classification, organisation and summarisation; stronger models are reserved for validation, tracing and synthesis. The orchestration layer owns state, gates, budgets and hand-offs; the model at each stage performs one focused piece of reasoning.
Harnesses Need Memories
Context management governs what a stage sees during a single run. Retrieval-Augmented Generation (RAG) governs what the harness can draw on from previous runs. The distinction matters: a harness with good context management but no persistent knowledge base rediscovers the same facts on every run, burning tokens and missing the accumulated learning from prior sessions.
The author has built a RAG store covering previous research notes, blog posts, tool documentation, language-specific references and other relevant content—giving both the harness and the underlying model access to a growing institutional knowledge base. Alongside this, a feedback loop runs after each successful finding, feeding the result back into the baseline so subsequent runs can build incrementally on what came before rather than starting from scratch.
Key Takeaways
- A harness—not the model—is the primary driver of capability, cost and reliability in AI-assisted security research workflows.
- MCPs provide callable tools; the harness decides when and how they are invoked. Having a full MCP suite without an orchestration layer still produces inconsistent, hard-to-verify output.
- Each pipeline stage needs its own purpose-specific prompt. Reusing a single system prompt across stages conflates tasks that require fundamentally different reasoning modes.
- Context windows are a budget: retrieve only what the current stage needs, summarise noisy tool output, and design context management into the pipeline from the start.
- Model routing matters: cheap, fast models for classification and summarisation; stronger models for validation, tracing and synthesis. Reserve expensive reasoning for the stages where it actually matters.
- A RAG-backed knowledge store and cross-run feedback loop transform a single-use pipeline into a compounding research platform that improves with each engagement.
- The harness-kit (recon → hunt → validate → trace → report) provides a minimal but complete template for structuring your own pipeline.
Practitioner Recommendations
- Start with a published harness as a baseline — RAPTOR for static and dynamic C/C++ analysis, Anthropic’s reference harness for ASAN-verified crash PoCs, Audit or VVAH for language-agnostic broad coverage. Understand what each one does before building from scratch.
- Write per-stage prompts — the recon stage, the hunt stage, the validate stage and the report stage have different cognitive tasks. Conflating them in one prompt produces worse output than splitting them.
- Instrument context consumption early — track token usage per stage from the first run. Tools like TokenBurn help map subscription usage to effective API cost and identify stages consuming disproportionate budget.
- Build validation gates before expanding scope — a pipeline that generates many findings without a mechanism for challenging them quickly becomes noise. Add a contradiction-check stage (like Scrutineer’s
revalidate) before adding more finders. - Store structured artefacts between stages — JSON hand-offs between pipeline stages (not raw conversation history) make individual stages independently testable and replaceable without disrupting the rest of the pipeline.
- Invest in a RAG knowledge store — research notes, tool documentation, language references and past findings all belong there. Cross-run memory is what separates a one-shot pipeline from a compounding research capability.
- Validate independently of the model — the Python execution layer in RAPTOR can run completely without Claude Code involvement. Design your own harness so the tool orchestration logic can be tested separately from the AI reasoning, and enforce execution-verified findings (ASAN crash PoC, Z3 constraint check) wherever the target allows.
Conclusion
The useful part of an LLM-assisted research workflow is rarely the model itself—it is the structure around it. How work is split into stages, what context reaches each stage, which tools are available, how findings are challenged, and what is remembered for the next run collectively determine whether the output is actionable or noise. A good harness does not eliminate the need for human judgement; it provides a repeatable, inspectable way to apply that judgement at scale. The harness-kit template is a practical starting point, but the design decisions covered here—per-stage prompts, context budgeting, validation gates, model routing, and persistent RAG memory—are the principles that actually govern the quality of what comes out.
Original text: “Harnessing Harnesses – Climbing the LLM Hills” by Andy Gill at ZephrSec / blog.zsec.uk.