LLMGateway EnterpriseRouting
A centralized internal gateway that fronts LLM calls for an entire org — auth, cost tracking/chargeback, model routing/selection, PII redaction, audit logging — server-side infrastructure, distinct from the client-side ProviderAdapter pattern in AI/LLMIntegration.
Why we need this / what value this brings
Without a gateway, every team calling an LLM builds its own auth/cost-tracking/redaction logic — and sensitive data (device configs, customer data) can leak into a third-party model with no central control point to prevent it.
When to use this
As soon as more than one team/use case in an org is calling LLMs — centralizing early avoids having to retrofit governance onto a dozen ad hoc integrations later.
How to use or implement this
Route all LLM calls through one internal service that handles auth, per-team rate limiting and cost attribution, redacts sensitive fields before they leave the org's boundary, and logs every request/response for audit.
Research questions
- Scenario: pre/post diffs from 1000s of devices get sent to an Enterprise LLM Gateway for summarization — what does that gateway need to guarantee (rate limiting per team, cost attribution per use case, redaction of sensitive config data) that a direct provider call wouldn't?
- How does an LLM Gateway relate to Networking/APIGateway — same 'single entry point' idea, but with LLM-specific concerns (token accounting, model fallback) layered on.
Empty folder — drop notes, links, and findings here as you research.