Source: arXiv — 2026-08-03
Summary
LongHorizon-Harness reframes long-horizon agent execution as a task-state management problem instead of a single growing context window. It runs a Manage-Execute-Audit (MEA) loop — a manager that tracks task state and picks the next subtask, a fresh-context executor that performs it, and a read-only auditor that independently verifies the resulting environment state before the loop advances. The approach targets a specific failure mode: agents whose task execution, task state, and self-assessment all live in the same growing context, letting incorrect self-assessments silently propagate into later decisions.
Key Takeaways
- The core insight is separation of concerns: the manager never executes, the executor never judges its own success, and the auditor never acts — only verifies against the live environment, which stops bad self-assessments from compounding across a long task.
- Each executor subtask runs in a fresh context rather than an ever-growing one, so errors and irrelevant history don't accumulate into the state the manager reasons over.
- Reported gains on Qwen 3.7-Plus: WeaveBench 51.8% to 80.7%, Terminal-Bench 2.1 69.7% to 77.2%, OSWorld 2.0 2.8% to 8.3% — all with the same underlying model, isolating the harness as the source of improvement.
- Claude Opus 4.7 improved from 20.0% to 34.3% on an OSWorld 2.0 subset under the same harness, suggesting the gains aren't specific to one model family.
- Code and the harness itself are public on GitHub (AMAP-ML/LongHorizon-Harness), with native integration for Claude Code, Codex, and OpenClaw, plus desktop/CLI computer-use support.
Reel Script
Hook (18s, ~40 words) An AI agent given a long task will happily tell you it succeeded when it didn't — because the same context window that's doing the work is also grading the work. One team just rewired that, and it nearly tripled a benchmark score.
Core Concept (75s, ~165 words) Long-horizon agents fail for a boring reason: everything lives in one conversation. The agent executes a step, then judges whether that step worked, using the same context that's already full of its own assumptions and mistakes. It's like grading your own exam while still holding the crib sheet you cheated with — errors don't get caught, they get inherited into the next decision. LongHorizon-Harness splits that single role into three. A manager holds the task state and decides what's next, but never touches the environment. An executor does the actual work, but starts every subtask with a clean, fresh context — no baggage from earlier steps. And an auditor, which can only read and verify, checks the environment directly against the state the manager holds, before anything moves forward. No component can both act and grade itself. That's the entire trick — and it's a workflow pattern, not a smarter model.
Hands-On (60s, ~140 words) The numbers make the case. Same model, Qwen 3.7-Plus, wired into this harness instead of a standard agent loop: WeaveBench success jumped from 51.8% to 80.7%. Terminal-Bench 2.1 went from 69.7% to 77.2%. OSWorld 2.0 — a notoriously hard desktop-automation benchmark — went from 2.8% to 8.3%, nearly triple, even though the absolute number stays low, which tells you how far long-horizon desktop tasks still have to go. And it wasn't just one model: Claude Opus 4.7 went from 20.0% to 34.3% on an OSWorld 2.0 subset under the identical harness. Nothing about the underlying model changed in any of these comparisons — only how task state, execution, and verification were separated. The code's public on GitHub, with Claude Code and Codex integrations already wired in.
Takeaway (25s, ~55 words) This is the most convincing evidence yet that agent harness design matters as much as model quality for long-horizon work. If you're building agents that run more than a few steps, the question isn't "which model" — it's "does anything independently audit what the agent claims it did." Go check whether yours does.