Source: arXiv (Microsoft, Fudan University, Zhejiang University, University of Edinburgh) — 2026-08-18
Summary
Agent Lightning v1.0 lets you plug almost any existing agent harness — the code that manages an agent's tools, context, and control flow — into a reinforcement-learning trainer without rewriting the agent. It works through a lightweight LLM endpoint proxy (about 3,500 lines of code) that intercepts the harness's normal request-response traffic and turns it into training signal, rather than making the trainer own the environment loop itself. On SWE-bench Verified, this let the team take Qwen3.5-9B from 41.8% to 56.4% using only 6,000 training examples.
Key Takeaways
- Calls the paradigm "harnessed agentic RL": the deploy-time harness — not the training engine — owns tool calls and environment interaction, while the trainer only ever sees sequences of LLM request/response pairs through the proxy.
- Reported gain: Qwen3.5-9B on SWE-bench Verified goes from 41.8% to 56.4% (+14.6 points absolute) after RL fine-tuning with just 6K examples and modest compute.
- The approach has to solve real plumbing problems most RL setups skip: retokenization across harness and trainer, merging samples from parallel agent runs, advantage calculation, and loss normalization.
- The original Agent Lightning approach has already been adopted by other RL frameworks (verl Uni-Agent, AReaL 2.0, slime, Polar), and the project is open-sourced at github.com/microsoft/agent-lightning.
Reel Script
Hook: Every AI coding agent runs inside a "harness" — the wrapper that manages its tools and memory. Retraining that agent usually means throwing the harness away and rebuilding the whole loop. Microsoft just found a way to keep it.
Core Concept: Reinforcement learning normally requires the training system to control every step an agent takes — which tool it calls, what it sees back — so it can compute rewards. But production agent harnesses (like Claude Code or OpenHands) already own that control loop, wrapping the model with tools, memory, and planning logic. Agent Lightning v1.0 doesn't fight that. It sits as a proxy in front of the model endpoint the harness already calls, quietly recording every request and response as they happen, then reconstructs a trainable trajectory afterward. The harness never knows it's being trained on. This "harnessed agentic RL" flips the usual assumption: the harness keeps driving, the trainer just watches and learns from the transcript.
Hands-On: The proxy layer is roughly 3,500 lines of code sitting between the harness and the model. Underneath it, the team had to solve unglamorous but critical problems: retokenizing text that the harness and the trainer's tokenizer don't agree on, merging partial samples from agents running in parallel, correctly computing advantage estimates when trajectories interleave, and normalizing loss so long and short episodes don't skew updates unevenly. The payoff shows up in a single concrete number: training Qwen3.5-9B this way on SWE-bench Verified — real GitHub issue-fixing tasks — moved accuracy from 41.8% to 56.4%, a 14.6-point jump, using only 6,000 examples.
Takeaway: If you're building or fine-tuning coding agents, this is the first practical blueprint for training the harness-plus-model system as a whole instead of picking one or the other — and a 14-point SWE-bench gain from 6K examples is a real efficiency signal, not a toy result. Worth cloning the repo before you design your next RL pipeline from scratch.