TechNewsFirst

Disclosure up front: this outlet is owned by Fedago, which builds and operates AIDE, a platform for running agents that write and ship code. That gives us a direct stake in the question this paper asks. We're disclosing it instead of pretending to write from neutral distance, and we're not using this paper to claim our own systems are ahead of anyone's, because we have no independently verifiable, benchmarked evidence that they are. That would be exactly the kind of unfalsifiable, self-serving claim this outlet exists to call out in other people's coverage.

With that said: here is a paper worth taking seriously if you build, or just use, coding agents.

Most coverage of coding agents treats the model as the whole story. GPT-whatever scored X% on SWE-Bench, Claude-whatever scored Y%. What that framing skips is that almost none of an agent's real-world behavior comes from the model alone. It comes from the harness wrapped around it: the loop that decides when the agent stops and thinks, what it's allowed to do at each step, and what it's allowed to remember as a task runs long. Anyone who has actually run one of these things for more than a toy demo knows the harness is where most of the engineering time goes, and also that most of it is done by feel. Summarizing the conversation "seems to help." Giving the model more tools "seems safer." Nobody has been testing these instincts against each other in a controlled way, because doing so is expensive and unglamorous.

That is what nine researchers did in "An Empirical Study of Harness Design for Coding Agents," posted to arXiv on September 17, 2026. Run-Ze Fan, Zihao Zhang, Simin Ma, Yebowen Hu, Shouju Wang, Kaiqiang Song, Fei Liu, Hamed Zamani and Xiaoyang Wang built a coding harness with a fixed execution loop, then varied three components independently: planning, the action space (which tools the model is allowed to use), and context management (what stays in the model's working memory as a task drags on). They ran this across four models on two established benchmarks, SWE-Bench Verified and Terminal-Bench 2.1, in 176 separately configured runs. That is an unusual amount of rigor for a question the field mostly settles by leaderboard vibes.

What they found, according to the paper's own summary of its results:

Context management matters more as the context window gets tighter, and it helps mostly by keeping the agent from blowing past its own memory limit and failing outright, not by making it reason any better. That is a deflating finding in the best way: the fix isn't a smarter agent, it's an agent that doesn't crash.

The best-performing approach chains two unglamorous tricks: strip out irrelevant material by fixed rule first, then have the model summarize whatever is left. A fancier version, one that lets the model "undo" a deletion and recover something it threw away, added complexity that the models mostly never touched and bought no accuracy at all.

Planning steps help weaker models get more tasks right. For stronger models, planning mostly just lowers the bill, with little change to how often the task actually gets done. A capable model doesn't need to be told to think before it acts; it just costs more if you don't remind it.

And the right toolset depends on how good the underlying model already is with raw commands. Models that are shaky with bash do measurably better when handed a curated menu of predefined tools. Models that are strong with bash do just as well, or better, with a bare terminal and no training wheels, at meaningfully lower cost.

This paper did not arrive in isolation. In the same stretch of 2026, at least three other preprints have circled the identical question from different angles: a source-code study of eleven real coding-agent systems ("Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents"), a paper on automatically evolving a harness through observability data ("Agentic Harness Engineering"), and one asking how harness design interacts with a model's own post-training ("The Interplay of Harness Design and Post-Training in LLM Agents"). None of that is coordination; it's a field converging on the same realization at the same time, which is itself the tell. The loud public argument is still "which model is smartest." The quieter, more consequential argument happening in these papers is "what surrounds the model, and does it turn that model's raw ability into a working result or waste it."

None of this should be mistaken for a finished, settled account. It's worth sitting with what a study like this can and can't tell you before treating any single number in it as gospel, which is what the honest-disagreement section below this article is for.

The house view here has always been that generative AI is genuinely revolutionary and, left alone, not smart enough to be trusted unsupervised, that a model working solo would be fired the same day it was hired. This paper is that argument with receipts, just narrower and more specific than the slogan suggests. A model that silently overflows its own memory on a long task, that can't distinguish useful history from noise without help, whose planning stops adding value the moment it's already competent, is a model whose failures are being caught by the scaffolding around it, not by the model itself. That's not an insult to the models. It's a fairly precise map of exactly which unsupervised failures the harness exists to catch, which is a more useful thing to know than another leaderboard score.