Source: arXiv (University of Illinois Urbana-Champaign / Meta AI) — 2026-08-05
Summary
EvoHarness-RL gives long-horizon LLM agents an external memory called "Belief, Progress, and Experience" (BPE) that lives outside the model's context window but is still readable/writable by the policy — instead of everything piling up inside an ever-growing prompt. Training happens in two stages: supervised fine-tuning first teaches the agent the harness's action space (when and how to call it), then reinforcement learning teaches it when to actually invoke the harness versus skip it. On ALFWorld with a Qwen3-8B backbone, this reaches 96.9% task success, and two interesting behaviors emerge during training on their own: the model calls the harness less over time as it internalizes recurring patterns ("harness annealing"), and the state itself gets compressed into a compact, task-adaptive form instead of growing unbounded ("harness evolution").
Key Takeaways
- The core architectural move: separate "Belief, Progress, and Experience" (BPE) state from the model's active context window, but keep it policy-facing — the agent explicitly learns to read and write it, rather than everything being crammed into one growing prompt.
- Training is two-stage: supervised fine-tuning first teaches the harness's action space (when/how to call it), then RL teaches the actual decision of when invoking the harness is worth it versus skipping it.
- Result: 96.9% task success on ALFWorld with a Qwen3-8B backbone — a strong number for an 8B model on a long-horizon task benchmark.
- "Harness annealing" — an emergent training dynamic where the model calls the harness less frequently over time, because it has internalized recurring harness patterns into its own weights.
- "Harness evolution" — the progress/experience state itself gets consolidated into a compact, task-adaptive substrate during training, rather than growing unbounded the way naive context accumulation would.
- Together these two dynamics suggest the harness isn't just a crutch — it's training signal that the model partially absorbs, similar to how a person stops consulting a checklist once the procedure becomes habit.
Reel Script
Hook (18s)
An 8-billion-parameter model just hit 96.9% task success on a long-horizon agent benchmark — not by getting a bigger context window, but by learning when to stop relying on its own memory system.
Core Concept (95s)
Long-horizon agent tasks — think multi-step tasks like navigating a simulated house to find and use an object — have a nasty problem: the more steps an agent takes, the more its context window fills up with history, and eventually either the window overflows or the important stuff gets buried in noise. EvoHarness-RL's answer is to give the agent a separate memory outside the context window, called Belief, Progress, and Experience state, or BPE for short. Think of it less like the agent's short-term working memory — the context window — and more like a notebook it can deliberately flip open, write a summary into, and flip closed. Crucially, this notebook isn't just a passive log the system maintains behind the scenes — the model itself learns to decide when to write to it and when to read from it, as part of its policy. Training happens in two stages. First, supervised fine-tuning teaches the model the mechanics — here's the notebook, here's how you write in it, here's how you read from it. Then reinforcement learning kicks in and teaches the harder skill: judgment. When is it actually worth stopping to consult or update the notebook, versus just continuing on momentum? That second stage is where the real capability gain comes from, because judgment about when to use a tool is harder to teach than the mechanics of the tool itself.
Hands-On (110s)
Here's the part that's genuinely surprising: two behaviors emerged during RL training that nobody explicitly programmed in. The first is what the researchers call harness annealing — over the course of training, the model starts calling its own memory system less and less often. That's the equivalent of a new employee checking the manual every five minutes in week one, and by month three just knowing the procedure cold and rarely opening it. The model is internalizing the recurring patterns the harness used to have to supply externally, folding them into its own weights. The second is harness evolution — the actual content of the Belief-Progress-Experience state doesn't just keep growing the way a raw transcript would. It gets consolidated, compressed down into something compact and adapted to the specific task, instead of ballooning unboundedly. Put those two together and you get a system that starts out leaning heavily on external scaffolding and, through training, both needs that scaffolding less and keeps what it does need leaner. On the benchmark side, this whole setup pushed a Qwen3-8B backbone to 96.9% success on ALFWorld, a long-horizon embodied task benchmark — a strong result for a model that size.
Takeaway (25s)
The real lesson here isn't the 96.9% — it's that giving an agent external memory it can learn to use, rather than just more context, is what lets a small model punch above its weight on long tasks. If you're building agents that run for many steps, invest in a harness the model can learn to rely on less over time, not just a bigger window.