A large language model gateway receives model requests and routes them to an approved provider or hosted model. It can normalize application programming interfaces, manage credentials, record usage and apply policies for limits, privacy and reliability.
Gateways make model choice easier to change without rewriting every product. They can also expose latency, cost and quality information, but evaluation and routing policies must remain current as providers and model behavior change.
Acronyms and aliases
LLM gateway acronymmodel gateway variant
Related terms
Frequently asked questions
What does a large language model gateway control?
It can control provider selection, credentials, request formats, rate limits, logging, fallback, cost and policy enforcement.
Is a large language model gateway the same as a model provider?
No. The gateway sits between applications and one or more providers or hosted models and presents a shared access layer.
