Source: Google Developers Blog — 2026-08-06
Summary
Google announced on August 6, 2026 that Agent and Model Evaluations in its Gemini Enterprise Agent Platform are now generally available, giving teams two built-in ways to grade agent behavior: code-based metrics (Python functions for exact-match, JSON-shape checks, or anything else expressible in code) and LLM-as-a-judge metrics (custom criteria, rating scales, and a chosen judge model). Evaluations can run locally against any model from any provider, or server-side against Model Garden models — explicitly including both Gemini and Anthropic models — folding evaluation into the same platform teams already build and deploy agents in, instead of leaving it as a bolted-on side project.
Key Takeaways
- Two distinct, natively supported metric types: deterministic code-based checks (exact match, JSON schema validation, arbitrary Python) for things with a clear right answer, and LLM-as-a-judge checks (custom rubric plus rating scale plus judge model) for things that need qualitative judgment.
- Local evaluation runs support any model from any provider — teams aren't locked into judging only Google models, even inside a Google-built platform.
- Server-side evaluation runs support Model Garden models, explicitly including Anthropic models alongside Gemini — a notable cross-vendor evaluation capability shipped inside a competitor's own platform.
- Shipping this as a first-class, general-availability platform feature — not a separate SDK, notebook, or third-party plug-in — signals Google is treating "how do you know the agent actually works" as core infrastructure, not an afterthought bolted on after launch.
Reel Script
Hook (~16s, 36 words) Most teams ship an AI agent, then bolt evaluation on afterward with a spreadsheet and a prayer. Google just made evaluation a first-class, built-in part of the same platform you build the agent in.
Core Concept (~70s, 155 words) There are two fundamentally different ways to grade an AI agent's output, and Google's platform now supports both natively. Code-based metrics are a strict grader with an answer key — did the output exactly match, is the JSON well-formed, does this number fall in the right range — anything you can express as a plain Python function. LLM-as-a-judge metrics are a grader with a rubric instead of an answer key: you write the criteria and a rating scale, and a second AI model reads the agent's output and scores it against that rubric, for things like "was this response actually helpful" that don't have one correct string to match against. You need both, because some things about agent behavior have a right answer, and some things require judgment — and treating both as first-class, built-in options instead of custom glue code is the actual feature here.
Hands-On (~60s, 135 words) Here's the workflow as it's actually built: you define a metric — either a code function or a judge rubric — and during development you run it locally against any provider's model, not just Google's. Once you're ready to test at production scale, you run the same metric server-side against Model Garden models, and Model Garden explicitly includes both Gemini and Anthropic models in that server-side evaluation path. So the same rubric you wrote to grade a Gemini agent can grade a Claude-based agent too, inside the same platform, with the same scoring pipeline. That local-and-any-provider versus server-side-and-Model-Garden split is the concrete architecture worth sketching: one evaluation definition, two execution paths, multiple model providers on both sides.
Takeaway (~20s, 48 words) Baking evaluation into the platform instead of treating it as a side project is the right call, and Google explicitly supporting judging of a competitor's models says evaluation infrastructure is going provider-agnostic faster than the models themselves. If your agent stack has no repeatable eval loop, this is the template to copy.