Source: arXiv — 2026-07-16
Summary
A new paper argues that "context engineering" — the instructions, tools, memory, retrieved knowledge, and guardrails an LLM agent is given — is not just vague best practice but an independently measurable leading indicator of whether that agent will actually be reliable in production. The authors build ProofAgent-Harness, an open-source evaluation infrastructure that scores an agent's context across seven specific axes — role clarity, guardrail coverage, instruction consistency, tool-schema quality, grounding sufficiency, injection hardening, and token efficiency — using multi-juror consensus scoring, deliberately kept isolated from behavioral outcome metrics. That isolation is the point: it lets the paper test whether context quality predicts downstream reliability, rather than just quietly correlating with it after the fact.
Key Takeaways
- The central claim: agents don't fail in isolation — when instructions are muddled, tool schemas are ambiguous, or guardrails are thin, the agent drifts, hallucinates, misuses tools, ignores constraints, or becomes vulnerable to prompt injection. The failure traces back to context, not just "the model got it wrong."
- ProofAgent-Harness scores context on seven concrete axes: role clarity, guardrail coverage, instruction consistency, tool-schema quality, grounding sufficiency, injection hardening, and token efficiency — each a specific, checkable property rather than a subjective vibe.
- Scoring uses multi-juror consensus (multiple independent judges scoring the same context, reconciled into a consensus score) rather than a single LLM-as-judge call, which is meant to reduce the noise and bias of any one evaluator.
- The methodological choice that matters most: the context score is computed completely separately from behavioral/release metrics. That separation is what lets the authors claim context quality genuinely predicts reliability, instead of the score being reverse-engineered from outcomes it's supposed to predict.
- ProofAgent-Harness is open-source, meaning teams can plug their own agent's system prompt, tool definitions, and guardrails into the same seven-axis rubric rather than relying on gut feel about whether their context engineering is "good enough."
Reel Script
Hook (~18s, ~40 words): When your production agent goes off the rails — ignores a rule, calls the wrong tool, gets prompt-injected — the instinct is to blame the model. A new paper says: check the context first, because that's usually where the failure actually starts.
Core Concept (~65s, ~145 words): "Context" here means everything you hand the agent besides the raw model — its system prompt, the tools it can call and their schemas, whatever memory or retrieved documents get stuffed into its window, and the guardrails meant to keep it in bounds. The paper's argument is that context quality isn't just a fuzzy best practice, it's something you can actually score, and that score predicts whether the agent misbehaves downstream. So they built an evaluator, ProofAgent-Harness, that grades context on seven separate axes — things like whether the agent's role is unambiguous, whether tool schemas are well-specified, whether instructions contradict each other, and whether the setup resists prompt injection. Multiple independent judges score each context and their verdicts get reconciled into one consensus number, instead of trusting a single AI judge's opinion. The trick that makes this rigorous rather than circular: they compute that context score completely separately from whether the agent actually succeeded or failed, then check afterward whether the two line up.
Hands-On (~55s, ~125 words): Picture the seven axes as a checklist you could literally run against your own agent's system prompt before shipping it: role clarity — does the agent know exactly what it is and isn't supposed to do; guardrail coverage — are the constraints actually enforced, not just mentioned; instruction consistency — do any two instructions contradict each other; tool-schema quality — can the model actually tell your tools apart from their descriptions; grounding sufficiency — does it have enough real data to avoid guessing; injection hardening — can untrusted input hijack it; and token efficiency — is the context bloated with stuff that dilutes the important parts. That's a concrete audit you can sketch as a seven-box scorecard and run against your own prompts today, and the harness itself is open source.
Takeaway (~22s, ~50 words): If your agent keeps misbehaving in ways that feel random, the fix is probably not a bigger model — it's auditing your context against something like these seven axes. Pull up ProofAgent-Harness and score your own system prompt before your next agent ships.