Source: arXiv — 2026-08-13
Summary
AutoDesign introduces a meta-harness optimizer that guides a code agent to recursively rewrite its own execution harness based on feedback from its own rollouts, applied to the task of paper-to-poster generation. On the paper's new PosterBench benchmark, the resulting agent autonomously executes 253 tool calls and 11 editing turns in roughly 40 minutes for under $3 in inference cost, scoring 78.32 — beating comparable systems by more than 7 points.
Key Takeaways
- The optimizer doesn't just tune prompts — it rewrites the agent's own execution harness (the code controlling how it plans, calls tools, and edits) based on how well previous rollouts performed.
- Benchmarked on PosterBench, a new long-horizon "turn a research paper into a poster" task, the optimized agent needed 253 tool calls and 11 editing turns to finish — real evidence of long-horizon autonomous operation, not a single-shot generation.
- Total cost per completed poster was under $3, with an end-to-end runtime around 40 minutes, giving a concrete cost/time baseline for long-horizon agentic design work.
- The 78.32 benchmark score beat comparable harness-optimization systems by 7+ points, suggesting the self-rewriting-harness approach outperforms static or prompt-only optimization for genuinely long-horizon tasks.
Reel Script
Hook (17s)
Most "AI agent optimization" just means tweaking a prompt. Researchers just built an agent that rewrites its own execution code between attempts — and used it to autonomously design a research poster for under three dollars.
Core Concept (90s)
Here's the distinction that matters: when people talk about optimizing an AI agent, they usually mean adjusting the instructions you give it — the prompt. AutoDesign does something different. It optimizes the harness — the actual code scaffolding that decides how the agent plans its steps, which tools it's allowed to call, and how it edits its own work — and it does this recursively, meaning the system runs the agent, looks at how the rollout went, and then rewrites the harness itself before trying again, not just the instructions inside it. Think of it like the difference between telling an employee "try harder next time" versus actually redesigning their workflow and toolkit based on what went wrong. They tested this on a new benchmark called PosterBench, which asks the agent to take a full research paper and autonomously turn it into a conference-style poster — a genuinely long-horizon task that requires reading, summarizing, laying out visual content, and iterating on the design, not a single generation step.
Hands-On (130s)
The numbers are the concrete part worth sketching out as a pipeline: teacher rollout runs, harness gets rewritten based on what worked and what didn't, agent re-attempts the task with the new harness, and the loop repeats. On the finished PosterBench task, the optimized agent made 253 separate tool calls and went through 11 rounds of self-editing before it was done — that's the kind of tool-call count you only see from a system that's genuinely operating over a long horizon, not one clever generation. All of that ran in about 40 minutes end to end, for a total inference cost under $3. And the resulting score, 78.32 on PosterBench, beat other harness-optimization approaches by more than 7 points — a meaningful gap on a benchmark, not noise. The architecture worth drawing is simple: optimizer watches agent → optimizer rewrites harness → agent re-runs with new harness → repeat, with the tool-call count and editing-turn count as the receipts that this is a real long-horizon loop, not a demo.
Takeaway (28s)
If your team is hand-tuning agent prompts and hitting a ceiling, the real lever might not be the prompt at all — it's the harness. A meta-optimizer that rewrites execution scaffolding based on rollout feedback is worth prototyping before you assume you need a bigger model. The cost and turn-count numbers here are a real baseline to benchmark your own long-horizon agents against.