The most likely answer isn't the best one

I gave a coding agent the same five-word task twice, word for word: write a simple tree serialization program in go. Same model both times, same everything I could hold constant. Both runs compiled, both round-tripped a tree correctly, and if you’d shown me either one in isolation I’d have called it a reasonable answer. But put them side by side and the gap is obvious. One of them I’d merge. The other I recognized the way you recognize a snippet you pasted out of a StackOverflow answer at 11pm and never opened again—correct, self-contained, and not something you’d want anywhere near a codebase you have to maintain.

The difference between the two wasn’t the model. It was about twenty lines of prompt, and that gap is what this post is about.

Two runs, same sentence

Run one got the task as a plain prompt and nothing else. It produced a single file: a Node type, a serialize and deserialize pair, and a main that builds a small tree and prints a round-trip, all in one package main blob. It wasn’t lazy about it, either. It reached for an N-ary tree rather than a binary one, invented a compact format like 1(2(5(),6()),3(),4(7(8()))), and wrote a genuinely careful parser—the kind that reports bad integers, unexpected characters, and trailing input along with their positions. A hundred and eight lines, and most of them earn their place. What it didn’t have was a module, a single test, or an exported function. Everything was lowercase and reachable only from the main sitting beneath it.

Run two got the same five words, but I didn’t hand them straight to the model. I routed the task through three throwaway sub-agents—a planner, an implementer, and a verifier—about twenty lines of prompt between them. Their exact definitions come in a moment; what matters first is what they produced.

What came back was a different kind of thing. Not a file but a project: a tree.go that was a library with exported Serialize and Deserialize, a main.go that ran a demo with an explicit round-trip OK/FAILED check, a tree_test.go covering the nil tree, a single node, a full round-trip, and idempotency, and a real go.mod tying it together. There was even a compiled binary sitting in the directory—tree-serial—because something in the loop had actually built it.

run one — plain prompt
  tree.go         108 lines: Node + serialize + deserialize + main, all inline,
                  unexported, no module, no tests

run two — planner → implementer → verifier
  go.mod          a real module
  tree.go         library: exported Serialize / Deserialize
  main.go         demo with a round-trip OK / FAILED check
  tree_test.go    nil · single node · round-trip · idempotency
  tree-serial     a compiled binary — it actually got built

The library at the center of it is small and unshowy. Where run one reached for an N-ary Node with a slice of Children, run two settled on a plain binary node and a preorder walk that emits null for the empty slots—and, crucially, exported the pair so another package could call it without copy-pasting:

type Node struct {
	Val   int
	Left  *Node
	Right *Node
}

func Serialize(root *Node) string {
	var tokens []string
	var preorder func(n *Node)
	preorder = func(n *Node) {
		if n == nil {
			tokens = append(tokens, "null")
			return
		}
		tokens = append(tokens, strconv.Itoa(n.Val))
		preorder(n.Left)
		preorder(n.Right)
	}
	preorder(root)
	return strings.Join(tokens, ",")
}

Same model, same sentence; the only thing I’d changed was the process wrapped around the words.

The scaffold, in twenty lines

The process wasn’t a framework. It was three Claude Code sub-agents, each a single markdown file with a few lines of role prompt, and me playing orchestrator between them by hand. Here’s the whole machine.

The planner reads but never writes. Its only job is to turn five words into a shape—files, steps, a definition of done—before a line of code exists:

---
name: planner
tools: Read, Glob, Grep
---
You produce implementation plans. Output: numbered steps, files to
touch, what each change should accomplish, and what "done" looks like.
Do not write code.

The implementer is the only one allowed near the filesystem, and it’s deliberately incurious—told to execute the plan rather than second-guess it:

---
name: implementer
tools: Read, Write, Edit, Glob, Grep, Bash
---
Follow the provided plan literally. If a step is ambiguous, flag it and
stop rather than improvise.

The verifier writes nothing and trusts nothing. Its entire definition is an instruction to assume the work is broken and go prove it:

---
name: verifier
tools: Read, Glob, Grep, Bash
---
You did not write this code. Your job is to find what's wrong with it.
Report concrete issues with file:line references; no praise.

There was no orchestrator tying these together—I drove the three stages by hand, addressing each invocation to one of the predefined agents and passing the planner’s plan into the implementer myself:

stage 1 → @planner       Decompose this task: write a simple tree
                         serialization program in go
stage 2 → @implementer   Execute this plan: {the planner's plan, pasted in}
stage 3 → @verifier      Audit this change against the original task:
                         write a simple tree serialization program in go

That’s the entire lever—three short role prompts, three one-line invocations, and a human passing notes between them.

Run one isn’t worse code

Here’s where the easy reading goes wrong, and it’s worth slowing down on. If you grade these two outputs the way you’d grade a coding exercise, run one arguably wins. It handles the harder data structure, and its parser is genuinely careful—it walks the string by hand and reports a bad integer, a premature end, or an unexpected character along with the exact position it choked on:

func deserialize(s string) (*Node, error) {
    ...
    parse = func() (*Node, error) {
        start := pos
        for pos < len(s) && s[pos] != '(' {
            pos++
        }
        if pos == start || pos == len(s) {
            return nil, fmt.Errorf("expected value at %d", start)
        }
        v, err := strconv.Atoi(s[start:pos])
        if err != nil {
            return nil, fmt.Errorf("bad int %q: %w", s[start:pos], err)
        }
        pos++ // consume '('
        n := &Node{Val: v}
        if pos < len(s) && s[pos] == ')' {
            pos++
            return n, nil
        }
        for {
            child, err := parse()
            if err != nil {
                return nil, err
            }
            n.Children = append(n.Children, child)
            if pos >= len(s) {
                return nil, fmt.Errorf("unexpected end")
            }
            if s[pos] == ',' {
                pos++
                continue
            }
            if s[pos] == ')' {
                pos++
                return n, nil
            }
            return nil, fmt.Errorf("unexpected %q at %d", s[pos], pos)
        }
    }
    ...
}

That reads cleanly, and it’s the kind of error handling you don’t usually get unasked-for. Run two, by contrast, cut the one corner run one didn’t:

// run two's Deserialize
val, _ := strconv.Atoi(tok)   // a malformed token silently becomes 0
n := &Node{Val: val}

Run one is not a weak model having an off day. It’s clean, it’s a little clever, and it works.

It’s just the most likely answer, and the most likely answer is a different thing from the best one. A strong model prompted the obvious way doesn’t reach for the best code it’s capable of writing; it reaches for the center of mass of everything “solve this in Go” looks like in its training data. That center of mass is overwhelmingly a self-contained, demonstrative snippet—built to be read and admired in isolation, not to be lived with. The instinct that produced the careful parser is the same instinct that skipped the tests: both make the snippet more impressive on its own, and neither has anything to do with whether the code survives being dropped into a real project.

The thing to hold onto is that the capability was never missing. Splitting a library off its main, exporting a clean API, writing a table of edge-case tests—all of that was already in the model, and run two used the very same model to do exactly it. Run one didn’t fail because it didn’t know how. It was never asked to optimize for anything beyond “produce a plausible Go program,” and the most plausible Go program is the centroid. That gap—between the answer a model is most likely to give and the best one it could give—is the whole subject. The question that actually matters isn’t whether the model can do the thing. It’s how much of what it can do your technique manages to pull out.

Why I’d keep run two

I was the only critic in this experiment, so this is my read, not a vote I collected. Sitting with the two outputs, I’d reach for run two without much hesitation—and not for the algorithm. What makes it the better artifact is everything that only starts to matter once code has to live next to other code: a module boundary, an API you can import without copy-pasting, and a set of tests that tell you why you’re allowed to trust the thing. And the tests aren’t ceremonial—they name the cases you’d actually worry about, the nil tree and the single node and the round-trip that has to come back unchanged:

// nil tree
if s := Serialize(nil); s != "null" { ... }

// single-node tree
single := &Node{Val: 7}
if s := Serialize(single); s != "7,null,null" { ... }

// idempotency: re-serializing a parse reproduces the input
input := "1,2,null,null,3,4,null,null,null"
if Serialize(Deserialize(input)) != input { ... }

Run one is something you read and admire for a second. Run two is something I could hand to a second developer who builds on it without first reading every line to be sure it does what it claims.

The instinct underneath—“would I want this in the codebase”—is really a quieter question: does this survive contact with someone who isn’t me. Run one doesn’t, and not because it’s wrong. It was simply never built to. Run two does, because the process that produced it was built around exactly that question.

The verifier did most of the work

There were three agents, but they didn’t pull evenly, and it’s worth being precise about which one mattered. The planner did real work: by forcing the task to be decomposed into files and acceptance criteria before any code existed, it’s the reason the output arrived already shaped like a project instead of a script. That’s where the library, main, and test split came from in the first place.

But the verifier is the one that bent the outcome, and its two-line definition above did it for an almost mechanical reason. Hand that agent a Bash tool and an instruction to find what’s wrong, and “run the tests” has no honest completion against a deliverable that has no tests. You cannot truthfully close out a step whose instructions are audit this and run the tests when there is nothing to run. So the tests got written, and once tests existed something had to compile and execute them, so the binary got built. Even the demo main carries the same fingerprint—it doesn’t just print the output and trust it, it checks the round-trip and says so out loud:

s1 := Serialize(root)
restored := Deserialize(s1)
s2 := Serialize(restored)

if s1 == s2 {
	fmt.Println("Round-trip OK")
} else {
	fmt.Println("Round-trip FAILED")
}

And you don’t have to take any of that on faith—run it and the proof is right there, the tests green and the demo echoing its input back unchanged:

$ go test ./...
ok  	tree-serial

$ go run .
1,2,null,null,3,4,null,null,null
1,2,null,null,3,4,null,null,null
Round-trip OK

The single largest visible difference between the two runs—proof that the code works, rather than an assertion that it does—traces directly back to one tiny agent whose only instinct is suspicion.

There’s a detail buried in here that I keep coming back to, because it complicates the tidy version of the story. Both prompts contained the word simple. Run one honored it with a single small file, which is a completely defensible reading of the word. The scaffold overrode it and shipped a multi-file module anyway. The planner and verifier roles, between them, pulled harder on the output than an explicit adjective in the task did—which is leverage, and also a warning. A process scaffold can quietly outvote an instruction you actually wrote down, and you want to be sure it’s voting the way you intended. For the record, run two genuinely lost something in the trade: run one’s nicer parser error handling. The scaffold optimizes for project hygiene, not for every local technical nicety. No free lunch, just a different optimum.

None of this is really about Go

I reached for Go and a tree because Go is gradeable. You can ask flat, mechanical questions of the output—does it compile, is it tested, is it structured—so the effect of the lever shows up on an oscilloscope instead of in a matter of taste. But nothing about the mechanism is specific to code, and that’s the part worth carrying away.

Every output a model produces has a modal version and a better-but-less-likely one sitting behind it. A first-draft paragraph and the revised one. A plausible-sounding analysis and the fact-checked one. The techniques that close the distance between them fall into two families, and it helps to hold both in view. The first is reshaping the objective: moving the model off the modal answer toward a target defined by quality rather than by likelihood. Acceptance criteria, decomposition, self-critique loops, and the cheapest member of the family—adversarial verification—all live here, and so does tool-grounding, the run it, test it, compile it move, because reality is the most honest verifier there is. The second family is search and selection: sampling more of the distribution and picking the best of what comes back, whether that’s best-of-N, a judge panel, or a tournament.

The throughline under both is the same. The distance between a model’s modal output and its real ceiling is large, it’s cheap to close, and you close it by redefining the target or searching the distribution—not by waiting for a better model to arrive.

You don’t have to wire this up anymore

It’s worth being honest about how much of this experiment was deliberately manual. I hand-rolled the scaffold: wrote the three agents myself and passed the planner’s output into the implementer by hand rather than letting anything orchestrate it. That was the point. I wanted the lever isolated so I could watch it move, with nothing else changing between the two runs.

But the loop I built by hand—decompose, implement, then adversarially verify—is increasingly something you switch on rather than something you assemble. Claude Code’s ultracode mode, for instance, fans work out across agents, has them check each other’s output, and treats verify-before-you-commit as table stakes rather than as a step you have to remember to request. The twenty lines of role prompts I wrote to run this experiment are, more or less, a manual version of a dial that’s now built in.

Which is the practical point sitting under the theoretical one. The gap between a model’s likely output and its best output has been there the whole time. Closing it used to require knowing to scaffold—knowing the technique existed and bothering to wire it up by hand. Increasingly it just requires knowing the gap is there, and turning the effort up to meet it.

For what it’s worth, this is n=1 per arm: a practitioner’s quick experiment, not a benchmark, and a vivid before/after rather than a measured effect size. I wasn’t trying to put a number on how far the lever moves. I just wanted to see whether it moved at all. It did.