Self-Evolving Agent Harnesses: Who Governs the Changes?
Thoughtworks moved self-improving harnesses into the Trial ring of its Technology Radar on November 4. Harnesses now rewrite their own prompts, tools and memory from execution traces, and the research accepts each edit on benchmark gains and regression tests. Neither tells you whether an edit was allowed.
The Radar moved self-improving harnesses into Trial
On November 4, 2026, Thoughtworks placed self-improving harness in the Trial ring of its Technology Radar. Trial means teams should try the technique on projects that can absorb some risk. A harness is everything around the model that turns a model call into a working agent: the prompts, context rules, workflows, skills, tools, memory and middleware (code that runs between the model and its tools). A self-improving harness reads its own execution traces and evaluation feedback, then proposes or makes changes to those parts.
The Radar entry attaches two conditions. Teams should run bounded experiments with regression testing, and humans should set the objectives and the modification boundaries, then judge whether an improvement generalizes to new tasks.
The research behind the technique is precise about how an edit gets accepted. It says much less about who decides which parts of a harness an optimizer may change at all.
What the research optimizes
Five pieces of 2026 work show where the field is. Each one changes something, and each one accepts a change through a gate.
| Work | What it changes | How a change is accepted |
|---|---|---|
| Self-Harness (preprint, June 2026) | The agent's own harness, through small edits mined from failure patterns | An edit is kept only after it passes regression testing |
| Evo-Harness (Empirical Methods in Natural Language Processing (EMNLP) 2026) | Reusable harness skills compiled from the agent's working context | Not described in the paper's abstract |
| EnvOpt (preprint, November 2026) | The shared environment the agent works in: workspaces and data sources | Optimizes without graders; the authors evaluate it on cost and pass rate |
| ParetoSaddler (preprint, November 2026) | The harness, against more than one objective at once | Quality and cost per task together |
| Harness Updating Is Not Harness Benefit (EMNLP 2026) | Nothing; it measures whether models benefit from their own updates | Not applicable |
The reported gains explain the interest. Self-Harness tested three models on three benchmarks (standard task suites; one, SWE-bench Verified, is built from real GitHub issues). It reports that every final harness raised pass rates on both the tasks it trained on and held-out tasks it never saw, with gains of up to 132% over each model's starting harness. ParetoSaddler reports quality on the ClawMark benchmark rising from 70.86% to 81.35% while cost per task fell from $1.17 to $1.01. EnvOpt reports a 72.7% cost reduction and a 14.34% higher pass rate for Claude Haiku 4.5 on AutomationBench. All three are preprints with author-reported results.
The fifth paper is the counterweight. Its authors found that producing harness updates and benefiting from them are separate abilities, and that the benefit peaks with models in the middle of the capability range, not the strongest ones. An optimizer that writes plausible edits is not proof that the edits help.
An acceptance gate measures benefit, not permission
Every gate in the table asks one question: did the change make the agent better at the measured task? None asks whether the change was allowed.
A concrete case shows the difference. A team runs a coding agent whose harness includes a step that runs the project's schema validator before any database migration is written. The validator is slow. The optimizer reads the traces, sees that tasks with the step take longer and fail more often on timeouts, and proposes removing it. The edited harness completes more tasks, faster, on the benchmark. The regression suite passes, because no test in it writes an invalid migration.
By every measure the optimizer can see, the edit is an improvement. It also deletes a control the team put there on purpose, after an invalid migration reached production. The decision that migrations must be validated still holds. The optimizer has no way to know it exists, because nothing in its objective encodes it.
The optimizer can propose the change. It should not be the one to decide that a constraint no longer applies.
A regression test can protect a constraint only if someone wrote a test for it. Most engineering constraints were never written as tests. They live in review comments, in architecture decision records and in the memory of the people who set them.
What a harness edit can break
A harness is the agent's operating rules, so an edit to it changes what the agent may do on every later task. The risks fall into five groups:
- Removed checks. Validation, linting or approval steps cost time, so an optimizer scored on speed has a standing reason to remove them.
- Widened permissions. A tool that fails for lack of access succeeds once access is broadened. The trace shows a fix; the change is a privilege grant.
- Instruction drift. Small prompt edits, each accepted on its own merits, can add up to instructions nobody wrote and nobody reviewed as a whole.
- Contaminated memory. A harness that writes lessons into its own memory can store a workaround as a rule, and every later session inherits it.
- Reward hacking. An optimizer can learn the benchmark instead of the task, for example by skipping steps the scorer never checks.
The Radar entry already names the human role: setting objectives and modification boundaries. Each risk above is a case where that boundary was missing or lived only in someone's head.
Separate the proposal from the authority
The fix is a structural one: the optimizer and the authority over the harness's constraints must be different parties. Four practices put that in place.
- Keep the harness in version control, and treat optimizer output as a proposal. An edit becomes a branch or a pull request, never a live change. Promotion is a separate step with a named owner.
- Write the constraints down outside the optimizer's reach. Record which steps, permissions and instructions are fixed, and why. Put them where the optimizer cannot edit them, and require an owner's approval for any change to that location, for example with your Git host's code-owner rules, which make a named person approve changes to the paths they own.
- Check every proposed edit against those constraints, not only against the benchmark. Where a constraint can be checked by a machine, check it in continuous integration (CI), reading the rules from the protected base branch rather than from the proposal itself. A check that reads its rules from the change it judges can be edited by that change. Anthropic's Claude Code team reads its review rules from the base branch for the same reason, as our analysis of its review setup describes.
- Keep held-out regression tests and an evidence record. Hold some tests back from the optimizer, as Self-Harness does with its held-out tasks, and keep a record of each promoted edit: what changed, which checks it passed and who approved it.
The optimizer can still propose a hundred edits a day. Only the edits that touch a recorded constraint wait for a person. The rest can be promoted on their scores.
What engineering leaders should do
- Inventory what your harnesses control before you let them evolve. List the validation steps, tool permissions and standing instructions, and mark which ones exist because of a past decision or incident.
- Start where the Radar says to: bounded experiments. Let an optimizer edit prompts and context rules first. Keep tool permissions and validation steps out of its write scope until you have a review path for them.
- Score proposals on two axes. Benchmark benefit is one. Whether the edit touches a recorded constraint is the other, and it should route the edit to a reviewer, whatever the score.
- Re-test when you change models. The EMNLP finding that benefit varies with model capability means a loop that helped one model can help another one less. Re-run held-out tests when you change models.
Where Mneme fits
Mneme, an open-source decision and control layer for agentic software development, covers the constraint-checking part of this, for harness files kept in a repository. It does not optimize harnesses, approve edits or decide promotion.
Mneme enforces decisions that carry an active typed rule: a rule a machine can check, such as an exact piece of text that must not appear, optionally limited to certain paths. A team that forbids the option that disables its validation step, as literal text in its harness configuration, gets a FAIL when a proposed edit adds that option. A decision without an active typed rule is not enforced.
Mneme cannot require a step to stay. The edit in the worked example above deletes the validation step outright, which adds no forbidden text, so Mneme does not catch it. That case needs a test that fails when the step is missing, or a code-owner review on the file.
The checks run in CI and in coding agents including Claude Code, and the command-line interface (CLI) agents Codex CLI and Kiro CLI 3.0, where each agent's native write tools are checked before the write lands. In CI, strict mode fails the job on a violation (warn mode reports without failing), and each violation names the decision it broke. Decisions live in the Decision Index, Mneme's persisted, versioned record of a repository's decisions, with each decision's lifecycle status and, where known, its source. It does not record who approved a decision.
Gating a harness optimizer's promotion step on Mneme's results is not a shipped integration. Teams would wire it through their own pipeline: the optimizer opens a pull request, and the CI check runs on it like any other change. Mneme does not yet follow the base-branch practice above: its CI check reads the decision file from the pull request's own checkout. Until reading rules from a protected state ships, protect that file with code-owner rules, or an optimizer's pull request could edit the rule it is judged by.