LLMIntegration

Calling LLM provider APIs directly: request/response shapes, streaming (SSE), rate limits, retries, and a provider-adapter abstraction so OpenAI/Anthropic/Groq/Ollama are swappable — or calling a unified router like OpenRouter that fronts many providers behind one API.

Why we need this / what value this brings

Without a provider-adapter abstraction, swapping OpenAI for Anthropic/Groq/Ollama means rewriting every call site — BYOK specifically requires this since the user picks the provider.

When to use this

As soon as more than one LLM provider needs to be supported, or you want to swap providers without touching call sites.

How to use or implement this

Define a single ProviderAdapter interface (send message, stream response, handle errors); implement one per provider; store the user's chosen provider+key client-side or per-session, never persisted server-side for BYOK.

Research questions

• What does a ProviderAdapter interface look like across 4 different providers' APIs?
• How do you stream tokens to a React client (SSE — see CommunicationPatterns/Protocols/ServerSentEvents) and cancel mid-stream?
• How do you handle provider-specific rate limits and error codes without leaking them to the UI?
• OpenRouter as an alternative to a hand-rolled ProviderAdapter: it's itself a multi-provider router (one API, many backend models) — when does using it outweigh the cost of routing through a middleman?

---

Empty folder — drop notes, links, and findings here as you research.