Source: arXiv — 2026-07-02
Summary
A 16-author paper introduces EvoPolicyGym, an agent-evaluation benchmark that shifts focus from one-shot code generation to "Autonomous Policy Evolution" — how well a harness-driven agent iteratively edits an executable policy under a fixed interaction budget across a suite of compact interactive RL environments. Rather than scoring a single output against a reference solution, it tracks trajectory-level diagnostics like budget allocation and how effectively feedback converts into tuning improvements.
Key Takeaways
- Core-16 suite of interactive environments tests iterative policy editing under a fixed budget, not single-shot generation.
- GPT-5.5 posts the strongest aggregate rank, placing top-two across all 16 environments in the reported results.
- Scoring uses hidden validation/held-out cases rather than a single reference solution, reducing overfitting to a known target.
- Provides trajectory-level diagnostics on how agents spend their interaction budget and convert feedback into improvement, which plain pass/fail benchmarks don't capture.