More than 60,000 open-source projects now ship an AGENTS.md, the plain-Markdown file that tells a coding agent how your repository is built, tested, and styled. The format is stewarded by the Agentic AI Foundation under the Linux Foundation, and more than twenty tools read it, including OpenAI Codex, Cursor, GitHub Copilot, Google Jules, Zed, Aider, and JetBrains Junie. Writing one is now the default first move on any serious repo. Almost nobody had checked whether it works.
A group at ETH Zurich did. Their preprint, first posted in February 2026 and revised in June, evaluated coding agents on real tasks with and without a context file and reached an uncomfortable conclusion, stated verbatim in the abstract: “providing context files does not generally improve task success rates, while increasing inference cost by over 20% on average.” That result held across different models, across different agents, and whether the file was written by a person or generated by an LLM (arXiv 2602.11988, a preprint, not yet peer-reviewed).
This is not an argument to delete the file. It is an argument to write a much smaller one, and to know which part of it is doing the work. That distinction is the whole of this guide.
What the study measured, and where the file failed
The researchers ran two settings. The first used established SWE-bench tasks from popular repositories paired with LLM-generated context files. The second was a new benchmark the team built to avoid the memorization problem: 138 issues drawn from 12 Python repositories that already carried developer-committed context files, filtered down from 5,694 real GitHub pull requests (ETH Zurich SRI Lab).
The split matters. In InfoQ’s breakdown of the results, the auto-generated files actually reduced task success by roughly 3 percent against giving the agent no file at all, while the human-written files bought a marginal gain of about 4 percent. Both added close to 20 percent to the bill. So the machine-authored context file, the one you get by asking your agent to “generate an AGENTS.md for this repo,” was worse than nothing, and the careful hand-written one barely cleared the noise.
The mechanism is the useful finding. The agents were not ignoring the file. The abstract is precise about this: “instructions in the context files are well followed by coding agents, repository overviews, although popular and recommended by model providers, are not helpful.” The agents obey. The problem is what we put in front of them. The single most recommended element, the tidy project overview at the top of nearly every AGENTS.md, is the part that does nothing. The model can already read the code. Telling it that your project is “a REST API built with FastAPI and PostgreSQL” restates what it will infer in seconds, and you pay tokens for the privilege on every run.
The part that earns its cost
The paper’s closing line is the design brief for a good context file: they are “useful for specifying non-standard coding practices.” Keep only what the model cannot work out from the repository in front of it. Cut everything else.
Keep:
- The test command with the flag nobody remembers, the one that skips the integration suite that needs a live database.
- The directory that looks hand-editable but is generated, so an agent that “fixes” it there loses the change on the next build.
- The deploy or migration step that fails silently when run out of order.
- The house rule that contradicts the language default: the linter config the team overrides, the import style you enforce against convention, the one library you have banned and why.
- Real security constraints: which secrets never get logged, which endpoints the agent must not call in a test run.
Cut:
- The project overview and anything that restates the README.
- The “you are an expert senior engineer” preamble.
- The exhaustive style list your formatter and linter already enforce on commit.
- Anything a competent developer would infer from opening three files.
The test is simple. For every line, ask whether an agent could have figured it out from the code. If yes, it is costing you tokens and buying you nothing, and on the LLM-generated evidence it may be actively steering the agent toward unnecessary work.
Write it like a runbook, not a README
In 20-plus years running IT operations, the documents that earned their place were never the ones that described the system. Anyone on the bridge could see the system. The runbook that saved a shift at 3 a.m. was the one that captured the single non-obvious step, the sequence that looked wrong but wasn’t, the switch that had to be flipped before the other switch or the failover would hang. The pages that restated the obvious got skimmed and ignored, and worse, they buried the one line that mattered.
An AGENTS.md is a runbook for a reader that never skims. That cuts both ways. The agent will read every word you write, every time, which is exactly why padding is expensive: it is re-sent and re-billed on essentially every invocation, so a bloated context file is a standing tax on each task rather than a one-time cost. This is where the discipline overlaps with the way prompt caching quietly decides your bill: a file the model reads on every call is the most reused block in your whole request, so its length compounds harder than almost anything else you send.
It is also the same lesson the context-rot research keeps returning: more context is not better context. A short file of genuine exceptions outperforms a long file of accurate summary, because the summary dilutes the signal the agent actually needs. The instinct to tell the model everything is the instinct to fight.
Measure it before you trust it
The authors do not tell you to abandon context files. They tell you to stop assuming. Their conclusion, verbatim: “any attempts to improve performance should be rigorously evaluated before deployment.” That is a specific, repeatable practice, not a vibe.
Take 10 to 20 real tasks from your own repository, ones with a known-good outcome. Run them through your agent with the current AGENTS.md and again with it removed, and compare two numbers: how many tasks completed correctly, and how many tokens each run burned. If the file is not clearly winning on the first number, it is losing on the second. This is the same argument this site has made about trusting your own evals over the leaderboard: the only benchmark that binds you is the one built from your work.
There is a real secondary benefit the researchers and their critics both noted, and it is worth keeping straight. Writing an honest context file forces a team to articulate knowledge that lived only in people’s heads, which helps the humans as much as the agents. That is a genuine documentation win. It is not an agent-performance win, and confusing the two is how you end up with a 400-line file that makes everyone feel organized and quietly slows every run.
Keep it small for one more reason
A context file is an instruction channel, and the study confirms the agent follows it faithfully. That is convenient until the file changes through a channel you did not intend. Anything that lands in your AGENTS.md, including a line slipped in through an outside pull request, is something your agent will treat as an instruction from you. That is the same review problem behind the plugin supply-chain flaw that hit four coding agents this month: the thing the agent trusts has to be governed like code, because it is code’s twin. A short file is easier to review, and a reviewer who knows every line is supposed to be a rare exception will notice when one is not.
The advice to “add an AGENTS.md” was never wrong. It was unfinished. The file works when it carries the handful of things your repository does differently and nothing else. With 90 percent of professional developers now running a coding agent weekly and 68 percent daily, the cost of a bloated file is not academic; multiply an unnecessary overview across that many runs and the 20 percent the ETH Zurich team measured turns into real money and slower tasks. Write the exceptions. Delete the summary. Then measure the difference, because on the current evidence you cannot assume it.