Google’s Harness² improves AI agents without touching the model

Harness² is an open source framework from Google Cloud AI Research that rewrites the working environment of an AI agent while leaving the model itself untouched. For each task it adjusts the prompts, guardrails, memory, skills and tool guidance around the model, tests those changes against each other and keeps only what works better in that comparison. A single improvement step added up to 11.5 points on WorkBuddy-Bench and 11.8 percentage points on Harvey’s legal benchmark LAB. The result that matters most for organisations: the open model GLM-5 with an improved harness outscored the stronger Gemini 3.5 Flash running on its standard harness on WorkBuddy-Bench.

This article explains how Harness² works, what the numbers do and do not show, what it costs and what it means for teams that build or buy AI agents.

What an agent harness is and why Google wants to rewrite it automatically

An AI agent has two parts: the language model and everything around it. That second part is the harness. It decides which instructions the model gets, which tools it may call, what it remembers and how it splits up work. Anyone who has worked on context engineering for Claude 5 knows how much a harness can change the behaviour of the same model.

Harness² splits the harness into eight files the system is allowed to edit:

• system prompt, the agent’s base instruction;
• guardrails, the rules and checks meant to stop mistakes;
• memory, what the agent keeps track of during a task;
• skills, fixed procedures for recurring work;
• tool guidance, when and how to use each tool;
• scripts, executable helper code;
• subagents, helpers the agent can delegate to;
• a middleware plugin that intervenes during execution.

Automatic harness improvement is not new. Meta-Harness, published in March 2026, searches offline for better harness code against a fixed set of training tasks. According to the authors of the Harness² paper, that approach breaks down in real deployments for two reasons. New tasks rarely come with correct answers or an automatic check that tells you whether an edit helped. And a harness tailored to one task often stops helping as soon as the next task asks for something different.

How Propose, Probe, Compose works

For every task, Harness² runs a contrastive step in three phases. An editor model does the rewriting. In the experiments that editor is Claude Opus 4.8.

1. Propose. The agent first runs the task with the current harness. Based on that reference run, the editor proposes two candidates. One targets general weaknesses such as planning, tool use, context management or delegation. The other targets the domain of the task, with the knowledge, procedures and constraints that apply there.
2. Probe. Both candidates run the same task under matched conditions. The system compares their steps and deliverables with the reference. The authors are careful here: the comparison guides the decision but does not prove the output is correct.
3. Compose. Based on the observed behaviour, the system merges the edits that were supported, revises edits that conflict, adopts one candidate or keeps the existing harness.

Teams with more compute can scale that step with a budget k. In parallel mode, k reference runs, k general and k domain candidates and k compositions run side by side, after which a selector picks the best anonymised deliverable. In sequential mode, k steps follow each other, each starting from the previous improved harness.

The square stands for a system that learns how to harness

The name Harness² refers to a second loop on top of the first. After each task, the system reviews the full improvement record and updates a library. Each entry holds an executable harness and an EXPERIENCE.md file describing the situation in which it worked, the lesson and the evidence. A lesson is only kept when the execution contrasts support it. If the evidence is thin, nothing is stored.

One design choice is worth noting. A new task does not inherit the harness of the previous task. It always starts from the base harness, but the editor can consult the library and reuse or adapt components. That keeps tailored fixes for one task from turning into noise for the next. The authors report that a library built on JobBench also helped on WorkBuddy-Bench, and that libraries built with a larger budget helped later edits more.

What the benchmark results show

Google tested Harness² on three benchmarks of professional work. WorkBuddy-Bench from Tencent includes code, web and office tasks, the three domains Harness² uses. JobBench has 130 tasks from 35 occupations, on which the best model in the original study reached only 45.9%. Harvey’s LAB tests legal work. The agents were Gemini 3.5 Flash and GLM-5, each running in two existing harnesses, OpenCode and Codex. Every result is a mean over three runs.

In three of the four cases the best baseline is Meta-Harness, which was optimised in advance on separate training tasks. For Gemini 3.5 Flash in Codex it is Harness Scaling at 73.8, and there no baseline beat the base harness. On JobBench, one step raised the score by 1.2 to 4.5 points and on LAB by 2.0 to 11.8 percentage points. At k=5, a further 3.0 to 4.2 points on JobBench and 2.6 to 6.7 points on LAB came on top.

Three findings stand out. On WorkBuddy-Bench, weaker agents gain more: GLM-5 added 10.9 to 11.5 points, Gemini 3.0 to 5.1. As a result, GLM-5 in OpenCode reached 75.7 after one step, above the 72.4 of unmodified Gemini. The authors put it plainly: “A weaker model with an improved harness can thus outperform a stronger one with its original harness.” Second, the alternatives are less stable. ReasoningBank fell up to 6.6 points below base on coding tasks, and Harness Scaling did worse than doing nothing in seven of twelve combinations. Third, the improved agent often needed fewer steps. Gemini in OpenCode went from an average of 25.4 to 21.9 steps per task on WorkBuddy-Bench.

The project page shows a concrete case. In a JobBench task that required reconciling invoices, incidents and posted credits into one checked workbook, the base agent left the sources unreconciled and never reopened its own output. The improved harness added a guardrail to verify the work, a reconciliation skill and more precise tool guidance. The score on that task went from 43.1 to 77.8.

What to keep in mind before quoting the headline numbers

Most coverage this week repeats the maximum gains. A few caveats put them in context.

• The editor is a frontier model. In the main experiments Claude Opus 4.8 rewrites the harness, even when the agent itself is GLM-5 or Gemini 3.5 Flash. Part of the weaker model’s gain may therefore come from a stronger model writing its working environment. The paper also reports cases where one model acts as both agent and editor, but the headline numbers come from the Opus setup.
• The maximums come from different setups. +11.5 and +11.8 are both GLM-5 in OpenCode, on different benchmarks. The smallest one-step gain was 1.2 points, for GLM-5 in Codex on JobBench.
• LAB is scored more leniently here. Harvey scores LAB all-pass by default: a task only counts when every criterion passes. Harness² reports “Partial”, the share of criteria passed. That is useful for seeing progress but it is not the official score.
• Improvement costs extra runs. Each step makes the agent perform the task several times. According to an analysis by wavect, that comes to four executions per step at k=1 and twenty at k=5, derived from the algorithm rather than measured, plus the calls for the editor, selector and grading. The paper includes a compute-versus-performance analysis but reports no costs in dollars.
• The baseline scores come from the Harness² paper itself. Meta-Harness predates JobBench and WorkBuddy-Bench, so the figures for Meta-Harness and the other methods are the Harness² authors’ own.
• Harness² does not win everywhere. On LAB, Meta-Harness scored 76.5 for Gemini in Codex against 75.6 for one Harness² step. And GLM-5 stays below Gemini’s base score on JobBench and LAB, even after improvement.

Trying the code, and what to lock down first

The code is on GitHub under Apache 2.0, at version 0.1.0. The README states that this is not an officially supported Google product. You need Linux, Python 3.12, uv, Node.js 20 or newer and the Google Cloud CLI. Models run through Vertex AI, and WorkBuddy-Bench needs Docker. harness2-demo --output runs/demo --k 2 runs both recursion modes without model credentials, using deterministic editors instead of an LLM.

Read the warnings in the README carefully. In this release the agent has unrestricted network access and Codex runs without its sandbox. Because benchmark rubrics are partly public online, the maintainers recommend restricting the network before you evaluate. The maintainers’ own experiment guide reports that, in a validation run, a Codex rollout on JobBench downloaded its task’s rubric without being instructed to. That says nothing about the results in the paper, but it shows why you should not point such a system at company data without isolation. Guardrails in an editable file are not a security boundary either: anything an agent must never do has to be enforced outside the agent.

What this means for teams that build or buy agents

You do not need to run Harness² to use its ideas. Three of them work just as well when people maintain the harness by hand.

• Compare on the same task. Test a new instruction or skill by running the same assignment with and without the change, and look at the behaviour, not only the final output.
• Separate general and domain fixes. A better planning rule belongs in the base harness. A rule about VAT rates belongs in a skill for that domain.
• Keep lessons with evidence. An EXPERIENCE.md file that records the situation and what each lesson showed is easier to manage than a system prompt that keeps growing. It is the same principle behind context-driven software factories, where every run makes the next one better.

The bigger point for procurement and architecture decisions sits in that library. If a weaker model with a better harness can overtake a stronger one, the harness becomes an asset you build up and keep, regardless of which model you use next year. As we argued before: models are rented, context is owned.

Frequently asked questions about Harness²

Does Harness² change the AI model itself?

No. The model weights stay frozen. Only the eight files around the model are edited, from the system prompt to the skills and scripts.

Do you need correct answers to use Harness²?

No. The system compares the behaviour of candidate harnesses on the same task and does not need an answer key or automatic verifier. That is why it works on unseen tasks, but the comparison guides decisions and does not prove the result is right.

Which models does Harness² work with?

Google tested it with Gemini 3.5 Flash and GLM-5 as agents, in the OpenCode and Codex harnesses, with Claude Opus 4.8 as editor. The code runs models through Vertex AI.

Is Harness² free?

The code is free under Apache 2.0. You still pay for model calls, and every improvement step makes the agent run the task several times.