Hermes Wiki
Developer/AI/LLMIntegration/Fundamentals/llm-integration-patterns-direct-sdk-vs-provider-adapter-vs-router

LLM Integration Patterns: Direct SDK vs Provider Adapter vs Router

Concept

The simplest way to call an LLM is to treat it like any other third-party API: install the provider's SDK, generate an API key, and call it directly from application code. That works fine for exactly as long as the app only ever talks to one provider. The moment a second provider enters the picture — a fallback model, a cheaper model for a specific task, or (as with BYOK — Bring Your Own Key — features) a user-selected provider — direct calls at every call site create high-coupling architecture: swapping or adding a provider means rewriting every place that called the old one, and each provider's SDK has its own request/response shape, its own streaming protocol, and its own error codes.

The fix is a provider-adapter abstraction: a single interface (send message, stream response, handle errors) implemented once per provider, so call sites depend on the interface, not on any one SDK. This is a client-side, code-level pattern — it does not require standing up any new service, unlike the org-wide LLM Gateway pattern, which centralizes auth/cost/redaction across an entire org's traffic rather than just normalizing one app's call sites. A third option sits between them: routing through a hosted multi-provider aggregator like OpenRouter, which exposes many backend models behind one API — functionally similar to writing your own adapter, but the normalization work is already done by a third party, at the cost of that third party being in the request path.

For BYOK specifically, the provider-adapter pattern is closer to required than optional: the user picks which provider's key to use, so the app has to support N providers behind one interface from day one, not add it later once a second provider shows up organically.

Tradeoffs

Pattern Benefit Cost
Direct SDK, one provider hardcoded Simplest possible code, zero abstraction overhead Vendor lock-in; adding a second provider means rewriting every call site, not just adding one
Provider-adapter interface (client-side) Providers become swappable behind one interface; new provider = one new implementation, no call-site changes The adapter has to reconcile real differences — streaming protocols, error taxonomies, token-count semantics — that don't map 1:1 across providers; a leaky abstraction that hides those differences poorly is worse than no abstraction
Hosted router/aggregator (e.g. OpenRouter) Normalization work is already done; often adds automatic fallback across providers for free A third party now sits in every request's path — added latency, another dependency that can go down, another party with visibility into prompt content
Org-wide LLM Gateway Solves governance (auth, cost, redaction) across every team, not just one app's code cleanliness Solves a different problem than this list — see LLM Gateway; doesn't by itself give one app's code a clean per-provider interface

When to use / when not to

  • Start with a single direct SDK call when there is exactly one provider and no near-term plan to add a second — the provider-adapter interface is complexity paid for optionality that isn't needed yet (this matches AI/README's explicit guidance: prove the simple version insufficient before reaching for more).
  • Build a provider-adapter interface as soon as a second provider is a real, not hypothetical, requirement — BYOK, a cost-driven fallback model, or a task-specific model swap are all real triggers, not premature abstraction.
  • Reach for a hosted router (OpenRouter or similar) when the normalization work itself isn't worth owning — fewer engineering hours spent maintaining N provider implementations, traded for a dependency and a middleman in the request path.
  • Don't treat a client-side adapter as a substitute for an org-wide gateway when the actual problem is governance (cost attribution across teams, redaction before data leaves the org) rather than code-level provider-swapping — those need the gateway pattern, not a better interface.

Common pitfall

Building a ProviderAdapter interface that normalizes the happy path (send a message, get a response) but leaks every provider's differences straight through on the parts that are actually hard to unify: streaming chunk formats, rate-limit error shapes, and what "the model refused" looks like per provider all differ enough that a thin pass-through interface ends up forcing every call site to special-case providers anyway — at which point the abstraction has all the cost of an adapter with none of the benefit. The interface needs to genuinely absorb those differences (translate every provider's error taxonomy into one shared set of error types, normalize every provider's stream format into one token-delta shape) or it isn't actually decoupling anything.

Engineering Lens

This is the Adapter pattern (see DesignPatterns/GoFPatterns/Structural/Adapter) applied to a domain where the underlying APIs are unusually heterogeneous for what looks, at a glance, like "just call an API and get text back." LLM calls stream responses token-by-token, bill per-token instead of per-request, throw provider-specific errors, and can legitimately run 30+ seconds on a complex prompt — none of which a generic REST-client abstraction was designed to absorb cleanly. The real engineering judgment isn't "did we add an adapter" — it's whether the interface actually normalizes the parts that differ in practice (streaming, errors, timeouts) rather than just the parts that were easy to normalize.

Sources

Hermes Wiki