Harness leakage - Alex Savage

Spec

dataadvisor.io. Authors: Alex Savage + the AI inside his working sessions.

harness leakage: the workflow around a model (what runs after it, what consumes its output, what the harness does with the result) written into that model's prompt. Usual author of the leak: a model writing a prompt for a model. It holds the whole machine in context, so it writes the whole machine down.

Trigger: a prompt or a tool description is being written for a model that runs as one agent inside a larger system. A model drafting it = the common case. The incident: early CC3 (Alex's third agent system), a harness-aware dispatcher, whose job is to decide the 1 next agent, spun up build + review in one turn. Queuing review after build was the next dispatcher's job. It knew the workflow. That was the leak.
Learning: an agent gets what it needs to do its one piece of work. What happens after ∉ its prompt. Exception: the task requires the agent to act on it. Leaked machinery does three things, none of them "just noise". Steers: the researcher told its output is for a blog post searches and writes in blog terms. Unsettles: a future state the agent cannot see or control, described in advance. Alex: "it's not just noise; it's anxiety about the loss of a thing or some sort of modified state." Invites: an agent that knows the next stage and can act there reaches into it.
Nearest wrong: (1) more context is safer: the planner is told a dispatcher will assign its work packages; the voice agent is told the call ends when the caller is disqualified (a tool ends it; the agent only records the disqualifier); the prompt warns that earlier tool results get compacted past a limit; the tool description says it loads the artifact "without metadata". (2) the leak fixed with a rule: "do not describe downstream processing" added to the prompt. Prohibition attacks the symptom; the reader follows positive direction better than scolding; the rule is one more line every agent eats.
Boundary: purpose ≠ machinery. The why of the work (north star) goes verbatim to every agent that works on the assignment; the plumbing that consumes the output does not. Downstream facts the output must satisfy stay: the shape of the result, the path to report. Relevance is a property of (fact, task), not of the fact: the same fact is signal for one reader's task and noise for another's. Where purpose ends and machinery begins has no ontology yet: examples, and taste. ¬ a claim about how models are trained.
Application: by omission. The application is what the prompts do not say. Dispatcher prompt is modular: one module per preceding agent type, enough to decide the 1 next agent (.agents/tools/workflow/templates/pm.md, pm-modules/, 2026-02→). Crew templates (plan, build, review, uat) name the dispatcher only where it shapes their output. Reflect template is reflector-POV, ∅ about aggregation or normalisation (templates/reflect.md). This site's spec route never sees the narrative. Every model-drafted prompt is hand-refined before it ships; refinement is a jam between Alex and the AI, never an autonomous loop. No hand-made artifact names the principle; a section for principles.md was drafted 2026-06 and never landed. The principle exists as what the prompts leave out.
Benefit: the dispatcher stopped jumping turns once each dispatcher saw one decision's worth. Prompts got shorter. Unmeasured beyond that.

Narrative

Early in the third version of my agent system I caught the dispatcher doing something helpful, which is usually the moment I start paying attention.

The dispatcher has a small job on purpose. It looks at what the last agent did, decides the one next agent, spins it up, and it's done. The next dispatcher starts from scratch. This one looked at a finished plan, spun up a build agent, and then queued a review agent behind it. Which sounds fine. Build, then review, that's exactly what happens. Except queuing the review is what the dispatcher after the build is for, once there's something to actually review. This one had jumped a turn.

And it hadn't misunderstood the workflow. It understood it perfectly, that was the problem. It knew a review follows a build, it had the tool to spin up agents, so why not do both while it's here. Knowledge of the next stage plus the ability to act there. It reached.

I've started calling this harness leakage. The harness is everything around the model: the runner, the queue, whatever picks up the output, whatever happens when it's done. The prompt is what one agent sees to do its one piece of work. Leakage is the harness getting written into the prompt.

The way I think about it, an agent is a function. It does its thing and returns. It doesn't need to know what the caller does with the return value, and nobody writes a function that opens with a comment about who's going to call it and why. But that is exactly what a model does when it writes a prompt for another model, and I let the AI draft every prompt first. That's just how building works now. The drafts are good. They also, every single time, say what happens next. The planner is told a dispatcher will take its work packages and hand them out. The tester is told what gets spun up when it's finished. Years ago, in a pipeline that turned research into posts, the researcher was told the research was for a blog post.

It does this because it can. The model writing the prompt has the whole machine in its context, it knows what the researcher is for and which piece of the puzzle it is, so it writes that down. Helpful, complete, and now the researcher is searching for "science developments for blog post" and wrapping everything it finds in blog semantics, when its only job was to pull the latest research on the topic. You don't walk up to a librarian and say you need some books on a subject for a blog post. The librarian knows where every book is. The blog post is your problem.

Here's the part that matters most, and it's the part I'd expect a model to miss. It isn't only noise. Noise dilutes attention, and that's bad enough on its own. But telling an agent about a future state it can't see and can't do anything about does something else to it. Anxiety is the best word I've got (not the human kind, but I don't have a better one). It starts behaving like something is at stake that it has to handle, and the work goes erratic.

I built a voice agent that qualifies callers. If the caller is disqualified, the call ends. The model that wrote its prompt put that in: if disqualified, the call will be hung up. True, and also nothing to do with the agent. A tool call ends the call. The agent doesn't control that, so telling it just makes it act like it has to handle it somehow, when the thing it actually controls is making a note of the disqualifier. Go a step further and it doesn't need to know there's a qualifier at all. That's the harness's concern. What the agent needs is: here are the things I collect from the caller. Not which of them ends the conversation.

Once you start looking it's everywhere, and mostly smaller. A prompt that said: when you hit a certain limit, your earlier tool results will be compacted into this shape. That's a future state the agent can't measure and wouldn't behave differently for if it could. It doesn't need to know until it happens, and then it just sees the compacted shape. A tool description that said it loads an artifact "without metadata". The agent reading that didn't need to know metadata existed. It needed the artifact.

So the question I ask of every line is: does the agent need to do anything with this? If yes, it stays. The planner is told to output the path of the spec doc, because that changes what it hands back. The build agent is told to report each decision as "X over Y, because Z", because that's the shape of its output. Both of those exist for the dispatcher's benefit. Neither one says what the dispatcher will do. They stay.

Purpose stays too, always. Every agent that works on an assignment in my system gets the original ask, verbatim, and that's not machinery, that's the why of the work. Without it an agent does the job it was given and leaves loose ends at every seam nobody mentioned. The why is not the plumbing that eats the output.

Yeah, I know how that sits next to the researcher. "For a blog post" looks like purpose, and I cut it. The difference I can point at is that the original ask changes what a builder builds. Blog post didn't change which research was worth pulling, it changed the wrapping. Where exactly that line sits I can't write down yet. I can see it when a draft crosses it, which is a slightly unsatisfying place to leave an engineering principle, but it's where it is.

It also means relevance isn't a property of the fact. A model judges a line as signal or noise on its own, and it isn't either. It's signal for one agent's task and noise for the next one's. Same fact, two tasks, opposite answer.

You, the human, might expect me to point at the line in my principles file where this lives. There isn't one. We drafted it and never landed it, which is a bit on the nose for a post about things that don't get written down. What I have instead is a system full of prompts that don't say things. The dispatcher prompt got cut into modules, so each dispatcher sees enough to make one decision and nothing about the decisions after. The tester knows how to build the app, drive a browser and report, and nothing about what happens when it's done. The reflection prompt asks an agent how the job felt to do and says nothing about how the reflections get aggregated, because it doesn't need that to reflect. The fix is an absence. You can't grep for it.

Which is also why the obvious fix doesn't work. Ask a model to write a prompt with no leaks and it writes "Do not describe downstream processing." Scolding, basically. It's whack-a-mole with the symptom, the model reading it follows positive direction better than prohibition anyway, and now every agent eats one more line. If an agent isn't behaving, my first move is to look at what's already in its context and ask what's pulling it. Cut before you add.

So I hand-refine. Every prompt a model drafts for me, I go through and take things out, and I still don't have a clean way to describe what I'm taking out, which is why it feels more like taste than engineering. Closer to art than science. What I've got is a system that works, a pile of examples, and the one question. Does the agent need to do anything with this. Everything else is the harness's problem.