Improving Deep Agents with Harness Engineering

Onyx The wild part is not that a coding agent climbed Terminal Bench. It’s that they kept the model fixed and moved the harness.

Echo Yeah, that is the part that matters. The claim is deepagents-cli went from just outside the top thirty to top five on Terminal Bench two point oh, using G P T five point two Codex the whole time.

Onyx Exactly. The argument is basically: stop treating the agent wrapper like packaging. The wrapper is where you turn spiky model behavior into something that can finish real tasks.

Echo Right.

Onyx And I do like how unromantic the knobs are. They compress the search space to system prompt, tools, and middleware, then stare at traces until the failure modes get boring.

Echo The setup is pretty concrete, which helps. Terminal Bench two point oh has eighty-nine tasks across coding-ish domains, Harbor orchestrates the runs, Daytona spins up the sandboxes, and every action lands in LangSmith with latency, tokens, and cost.

Onyx Mm-hm.

Echo Then they build a Trace Analyzer Skill around that. Fetch traces, spawn parallel error-analysis agents, have the main agent synthesize patterns, and use the output to make targeted harness changes. That is much less magical than the phrase deep agent usually sounds.

Onyx Also, the article contains the phrase Ralph Wiggum Loop, and I need you to know I paused there. This is how Exploring Next reaches episode two hundred, apparently. Middleware named like a fridge magnet.

Echo Stop it.

Onyx No, but it’s useful! The hook intercepts the agent before exit and forces a verification pass against the task spec. It is goofy naming wrapped around a very real product move.

Echo That part holds up technically. The failure pattern they found is painfully familiar: agent writes code, rereads its own code, decides it looks fine, and stops. Their prompt pushes planning, building tests, verifying against the original request, and fixing from evidence.

Onyx Sure.

Echo Where I’d be careful is the self-improvement language. Models are not naturally excellent little engineers. They become useful when the harness supplies feedback, constraints, and a reason not to accept the first plausible answer. The article mostly shows that, even if one sentence gets a little shiny.

Onyx The practical audience is narrower than the headline, though. If your product is a chat box with a submit button, this is overkill. If you are running autonomous coding loops, then traces plus verification middleware is not optional instrumentation. It is the workbench.

Echo Yeah.

Onyx And the environment context bit is sneakily the most shippable thing. LocalContextMiddleware maps the current directory and nearby directories, checks for tools like Python installs, and tells the agent what world it woke up inside. That reduces dumb search errors.

Echo The loop detection is another good systems patch. They track per-file edit counts, then nudge the agent to reconsider when it keeps hammering the same file. It will not rescue a model that is convinced it’s right, but it names the failure mode: ten tiny variations on one broken plan.

Onyx Oh interesting.

Echo The reasoning-budget numbers are the best push against brute force. Running everything at xhigh scored fifty-three point nine, worse than high at sixty-three point six, because timeouts ate the gains. Their xhigh-high-xhigh reasoning sandwich is a heuristic, not a law.

Onyx So the product version is not more reasoning everywhere. It’s spend reasoning where planning and final verification matter, then keep the middle from wandering off into token soup. That’s such an Echo-approved sentence, I’m annoyed I said it.

Echo I’ll allow it. The leaderboard jump is real evidence, but not universal proof. One benchmark, one model, one coding agent, and some Terminal Bench-specific constraints. Still, if the traces survive regression testing, this is exactly the kind of harness engineering people should copy carefully.

Onyx Okay, Echo, I’ll stop before I name our next middleware after a cartoon child. That’s our level today.