Agentic Retrieval Arrives for Amazon Bedrock Managed Knowledge Base

Source: AWS Machine Learning Blog — 2026-07-24

Summary

AWS added a new retrieval mode, exposed via the AgenticRetrieveStream API, to Amazon Bedrock Managed Knowledge Base. Instead of running one similarity search per query, it runs a planning loop driven by a foundation model: the model decomposes a complex question into sub-queries, retrieves evidence for each, judges whether it has enough to answer, and iterates until it does — all visible through a real-time stream of trace events showing each planning step, sub-query, and evaluation round. AWS positions the plain Retrieve API for simple factual lookups and AgenticRetrieveStream specifically for multi-part questions that a single retrieval pass tends to answer incompletely.

Key Takeaways

• Standard RAG retrieval runs one embedding search and hands whatever comes back to the model — it doesn't check whether that single pass actually covered a multi-part question, which is exactly where classic RAG quietly gives incomplete answers.
• AgenticRetrieveStream puts a foundation model in the loop as a planner: it breaks a complex query into sub-queries, retrieves against each, evaluates sufficiency, and decides whether another retrieval round is needed — a multi-step decision process instead of one-shot retrieval.
• The API streams trace events in real time — each planning step, each sub-query issued, each evaluation round — giving full visibility into why the system retrieved what it retrieved, without the caller having to build custom instrumentation.
• It generates a grounded answer by default in the same call (generateResponse=True implicitly), but can be set to return just retrieval results (generateResponse=False) for callers who want to run their own generation step.
• AWS's own guidance is explicit about when to use which: plain Retrieve for simple factual queries, AgenticRetrieveStream for complex, multi-document questions requiring several retrieval passes — it's an added tool, not a wholesale replacement for the simpler API.

Reel Script

Hook Most RAG systems answer "what were our Q3 numbers and how do they compare to last year's plan" by grabbing one batch of matching text and hoping it covers both halves of the question. AWS just shipped an API that actually checks its own work.

Core Concept Classic retrieval-augmented generation works like handing a research assistant one search query and whatever they find in five minutes — good enough for "what's our return policy," useless for a question with three parts buried in different documents. A vector database search — think of it as finding documents whose meaning is mathematically close to your question, not just matching keywords — returns one batch of results and stops there. Agentic retrieval changes the process, not just the search: a foundation model acts like a research assistant who actually reads the question, breaks it into its component parts, looks up each part separately, and — crucially — checks whether what it found actually answers the question before reporting back. If it doesn't have enough, it goes and looks again. That loop — decompose, retrieve, evaluate, repeat if needed — is the whole difference between "agentic" and "classic" retrieval here.

Hands-On The mechanism, as AWS describes it: call AgenticRetrieveStream instead of Retrieve, and instead of getting back one flat list of chunks, you get a stream of trace events showing the planning model's actual work — "here's the sub-query I generated," "here's what I found," "here's whether I judge that sufficient," repeated across however many rounds the question needs. That trace stream is worth building a mental diagram of: Question → Decompose into sub-queries → Retrieve per sub-query → Evaluate sufficiency → (insufficient? loop back to decompose again) → (sufficient? generate grounded answer). By default the API also generates the final answer for you in the same call, though you can turn that off and just take the retrieved evidence if you want to run your own generation step. AWS's stated guidance is to reach for this specifically on multi-part questions — not to swap it in everywhere Retrieve already works fine.

Takeaway If your RAG system is quietly failing on multi-part questions and you've been patching it with bigger context windows or manual query rewriting, this is the more principled fix — evaluate it against your actual failure cases before assuming a single retrieval call architecture is good enough.

Discussion

(No questions yet — ask follow-ups via a Claude Code chat session on this repo; answers get appended here.)