Source: Elastic Blog — 2026-08-18
Summary
Elastic published a five-takeaway retrospective on what it's learned building production architecture for AI agents, centered on the idea that retrieval has shifted from simple lookup to active reasoning — agents now need to dynamically inspect a query, determine what data is actually required, and iterate until they have enough information, rather than doing one static similarity search. The piece argues vector similarity alone is insufficient at production scale, and that hybrid retrieval, reranking, and access controls need to be integrated together, with context explicitly engineered and delivered at the right granularity and moment in the reasoning loop — since, unlike humans, agents carry no implicit domain knowledge of their own.
Key Takeaways
- Retrieval has shifted from a single static lookup to active, iterative reasoning: agents now inspect a query, determine what's missing, and search again — closer to a research process than a one-shot database query.
- Vector similarity alone is called out as insufficient for production agentic workloads; the piece argues for integrated systems combining hybrid retrieval (vector plus keyword/structured search), reranking, and access control together, not any one in isolation.
- Central framing: "if accuracy is the goal, context engineering is how you achieve it" — positioning context engineering as the primary lever for agent accuracy, above prompt tuning or model choice.
- Agents carry no implicit domain knowledge the way a human employee does — every piece of context has to be explicitly and deliberately provided at the right level of detail and the right moment in the agent's reasoning loop, or the agent simply doesn't have it.
- Elastic frames this as requiring "fundamentally different infrastructure" than search-driven (human-facing search box) systems were built for — a distinct architecture problem, not a tuning problem on top of existing search infrastructure.
Reel Script
Hook: If your AI agent keeps confidently getting things wrong, the fix probably isn't a better prompt — it's that nobody engineered what context the agent actually has access to at the moment it needs it.
Core Concept: A human employee walks into a new task with years of implicit context — company norms, past decisions, what "normal" looks like. An AI agent has none of that by default; it only knows what's explicitly placed in front of it at that moment. Elastic's argument is that as retrieval-augmented systems move from powering a human-facing search box to powering autonomous agents, the retrieval layer has to change jobs: instead of one static similarity search returning "here are some related documents," it needs to become an active, iterative process — the agent inspects what it's missing, searches again, and keeps going until it actually has what it needs, closer to how a researcher works than how a search engine works.
Hands-On: The concrete architectural claim is that vector similarity — finding documents whose embeddings are mathematically close to your query — isn't enough on its own at production scale. It needs to be combined with hybrid retrieval (blending vector search with traditional keyword and structured search, which catches exact-match cases vectors miss), reranking (a second pass that reorders initial results by relevance before they ever reach the agent), and access controls baked into the retrieval layer itself, not bolted on afterward. The underlying discipline this all falls under is context engineering: deliberately deciding what information reaches the agent, at what level of detail, and at which specific point in its reasoning loop — treated as its own engineering practice, not a side effect of having a good vector database.
Takeaway: If your RAG pipeline is "embed everything, do a similarity search, stuff the top-k results in the prompt," that's the exact pattern this piece argues doesn't hold up in production — the fix is treating what context reaches your agent, and when, as a deliberately engineered pipeline stage, not an afterthought bolted onto retrieval. Audit your retrieval layer for hybrid search and reranking before you blame the model for a context problem.