================================================================
Harness²: Teaching an Agent to Rewrite Its Own Operating Manual — A Source-Checked Reading
让 AI 自己改自己的「操作手册」：Harness² 核源解读
----------------------------------------------------------------
vertciti Data Room · Public Document
Date: 2026-10-20 | Version: v1
Sources: GitHub repository google-research/harness2; paper arXiv:2610.14224;
  project page harness2.github.io. All figures as reported in the paper;
  see source notes inline.
Classification: Public — contains no confidential information.
================================================================

----------------------------------------------------------------
ENGLISH
----------------------------------------------------------------

Introduction

In October 2026, the google-research organization open-sourced Harness², research code accompanying a paper by Google Cloud AI Research together with a Northwestern University researcher. The question it asks is a practical one: if the model weights stay frozen and you edit only the "operating manual" around the model — prompts, tool guidance, memory, skills — can the same model do better work? And can the model be trusted to edit that manual itself?

Source-checked points

1. What it is — and is not. The repository sits under Google's official research organization and is Apache-2.0 licensed, but its README states plainly that it is "not an officially supported Google product." It is research code: created on 2026-10-06, with no formal release yet.
2. The mechanism. The framework decomposes an agent's surrounding system into eight editable file components (system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, plugins). Improvement runs as Propose–Probe–Compose: propose candidate edits, actually run each candidate on the same task under the same conditions as a probe, then compose the successor from the observed behavioral differences. The refinement loop deliberately uses no rubric and no answer key.
3. A persistent experience library. After each task, the adopted changes and the lessons the evidence supports are written into an improvement library that later tasks can retrieve — while the previous task's full configuration is deliberately not inherited wholesale, so one-off task noise does not leak into the manual.
4. Reported results — and the paper's own warning. On WorkBuddy-Bench, a single improvement step lifted GLM-5 from 64.2 to 75.7, and a multi-step run took Gemini 3.5 Flash to 80.8 (figures as reported in the paper). Just as important is the built-in negative control: editing the manual by intuition, without probe runs, dropped scores from 73.4 to 63.2. In this paper, "evidence first" is not a slogan — it is the difference between gain and loss. The paper also notes its benchmark scores come from model-based judging, so they are best read for method and magnitude, not as absolutes.
5. The boundaries. The release supports two executor bases, OpenCode and Codex. Only the file layer evolves; the weights stay frozen, and the paper is explicit that the additional inference cost cannot be eliminated and the optimal budget split is still open.

The vertciti view

At vertciti, where we build a personal AI operating system for one, the operating manual is a first-class asset: prompts, tool guidance and experience notes are versioned separately, and every change is probe-tested before it merges. Harness²'s probe-driven method points the same way. Its fully autonomous "edit and adopt immediately" loop is where we stay conservative — what changes the manual should still be decided by evidence and by people.

Sources

- GitHub repository: https://github.com/google-research/harness2
- Paper (arXiv:2610.14224): https://arxiv.org/abs/2610.14224
- Project page: https://harness2.github.io

----------------------------------------------------------------
中文
----------------------------------------------------------------

导语

2026 年 10 月，google-research 组织开源了研究代码 Harness²，配套论文出自 Google Cloud AI Research 团队与西北大学研究者之手。它问的是一个很实际的问题：模型权重不动，只改模型外围的「操作手册」——提示词、工具说明、记忆、技能等文件——同一个模型能不能干得更好？以及，这份手册能不能交给模型自己来改？

核源要点

1. 真身与性质：仓库位于 Google 官方研究组织 google-research 之下，Apache-2.0 许可；但 README 明示「这不是 Google 官方支持的产品」。它是研究代码：2026-10-06 建仓，目前尚无正式版本发布。
2. 机制：框架把 agent 的外围系统拆成 8 个可编辑文件组件（系统提示、护栏、记忆、工具说明、子代理、技能、脚本、插件）。改进走 Propose–Probe–Compose 三步：提出候选改法 → 在同一任务、同一条件下真实试跑探针 → 按行为对比合成新版。精炼回路内刻意不使用评分标准与答案键。
3. 持久经验库：每个任务结束后，「被采纳的改法＋有证据支持的教训」被写入改进经验库，供后续任务检索复用；但上一任务的整套配置不被直接继承，以防把一次性的任务噪声带进手册。
4. 效果与反面证据（论文报告）：在 WorkBuddy-Bench 上，单个改进步骤后 GLM-5 得分由 64.2 升至 75.7，多步运行后 Gemini 3.5 Flash 达 80.8。同样重要的是论文自带的反面结果：跳过试跑、凭理解直接改手册，得分从 73.4 跌到 63.2——没有探针证据的「自优化」是负优化。
5. 边界：发布版只支持 OpenCode 与 Codex 两个执行基座；「进化」的仅是文件层，模型权重冻结；论文在局限部分明说额外推理成本不可消除、最优预算分配仍未定。论文亦自承其评测分数由模型评审打分、引用时只宜取方法与量级方向。

vertciti 视角

我们在为一个人构建个人 AI 操作系统的过程中，同样把「操作手册」当作一等资产管理：提示词、工具说明与经验记录分开版本化，任何改动先小规模试跑、再合并。Harness² 的探针式改法与这条纪律同向；至于它「改完即自动采纳」的自治闭环，我们保持谨慎——改手册的决定，仍应由证据和人来把关。

来源

- GitHub 仓库：https://github.com/google-research/harness2
- 论文（arXiv:2610.14224）：https://arxiv.org/abs/2610.14224
- 项目页：https://harness2.github.io

---