Source: Elastic Search Labs — 2026-07-21

Summary

Elastic added a native nvidia service type to its Open Inference API, letting Elasticsearch call NVIDIA-hosted, NIM-optimized models directly for text embedding, completion, chat completion, and reranking — configured with just an API key and a model ID, no custom integration code. The post documents specific supported models, including nvidia/llama-3.2-nv-embedqa-1b-v2 for embeddings and nv-rerank-qa-mistral-4b:1 for reranking, wired up through the same PUT _inference endpoint pattern Elastic already uses for other providers like Cohere and Mistral.

Key Takeaways

• One PUT _inference/{task_type}/{endpoint_id} call with "service": "nvidia", a model_id, and an api_key in service_settings is enough to register an NVIDIA embedding or rerank endpoint — no client library or custom REST integration needed.
• Named models: nvidia/llama-3.2-nv-embedqa-1b-v2 for text embedding and nv-rerank-qa-mistral-4b:1 for reranking, both NIM-optimized for retrieval workloads.
• Covers four task types through one integration path — text embedding, completion, chat completion, and reranking — so a RAG pipeline's retrieval and reranking stages can both live on NVIDIA-hosted models without separate plumbing.
• Fits Elastic's broader Open Inference API pattern (already supports Cohere, Mistral, Jina AI, and others), so switching providers is a config change on the service field rather than a code rewrite.

Reel Script

Hook If you've built a RAG pipeline, you know the retrieval and reranking steps usually mean stitching together two or three different SDKs, each with its own auth and its own quirks. Elastic just collapsed that into a single API call.

Core Concept Reranking is the step in a search pipeline where you take a batch of roughly-relevant results and have a specialized model re-score them for actual relevance to the query — think of it as a second, smarter pass after a fast-but-fuzzy initial search. Elastic's Open Inference API is a provider-agnostic wrapper: you register an inference endpoint once, pointing it at whichever model vendor you want, and every other part of Elasticsearch — search queries, ingest pipelines — just calls that endpoint by name. What's new is a first-class nvidia service type in that wrapper, meaning NVIDIA's NIM-optimized models for embedding and reranking are now just another service value, not a custom integration you have to write and maintain yourself.

Hands-On The setup is genuinely one API call: a PUT request to _inference/text_embedding/your-endpoint-name, with a JSON body naming "service": "nvidia" and, inside service_settings, your API key plus a model_id like nvidia/llama-3.2-nv-embedqa-1b-v2. Swap the task type and model ID and the same pattern registers nv-rerank-qa-mistral-4b:1 as a reranking endpoint. Once registered, both endpoints are callable from Elasticsearch's normal semantic search and rerank query clauses — the embedding model and the reranker sit behind the same interface as every other inference provider Elastic supports.

Takeaway My verdict: the code here isn't clever, and that's the point — the value is that swapping model providers becomes a config edit instead of a re-architecture, which is exactly the kind of infrastructure debt reduction that matters more than any single model's benchmark score. If you're running Elasticsearch for RAG, this is worth trying against your current embedding provider as a drop-in comparison. Follow for more on the plumbing that actually ships in production search systems.

Discussion