Model access, permissions, security.
All at the gateway.
Stop managing credentials in every application. Keep upstream credentials in the gateway, issue keys to apps, and run security checks in one place.
Get startedInspect what goes in.
Review what comes out.
Run input and output checks at the gateway. Credential patterns, code markers, and linked requests all flow into security event management so your security team has evidence to investigate.
Credential detection
If a request carries a password or key pattern, the gateway flags it. Your security team can review.
Output inspection
Check model replies for configured code markers and log match locations for review.
Security investigation
Connect rules, match locations, and request records in one place. No cross-system lookups to investigate a risk.
password=[REDACTED]<script [omitted]Security event
- Match details
- Rule · Source · Match location
- Action review
- Recorded matches and actual outcomes
- Request correlation
- Linked request records by request ID
Control access by organization and model.
Use gateway keys for applications and provider credentials for model services. Configure access by organization, role, and model, with separate permissions for resource management and application requests.
Explore teams & keysValidate policies for your use case.
Verify detection, event recording, and actual actions, including streaming and non-streaming coverage. See the security guide for code marker checks and event retention details.
Read about policy actionsConnect any model.
Expose one clean endpoint.
Configure providers, channels, and custom endpoints in one place. Your apps connect using a single standard API without rewriting integrations.
Explore models & channelsControl who calls what.
Without confusion.
Keep upstream credentials secure in the gateway. Issue dedicated keys to teams and apps with model whitelists, rate limits, and monthly quotas.
- Provider credential
- Gateway → Model service
- Gateway key
- Application → BOS API
Know your token costs.
No surprises.
See token usage and calculated costs by team, app, or key. Spot high-traffic services and monitor budget consumption in real time.
Costs are calculated from usage and configured pricing.
Explore usage & quotasWhat happens inside
a gateway request?
Authentication, quota checks, routing, protocol proxying, and logging happen automatically on the wire.
- 01Authenticate
Validate gateway key and model access permissions
- 02Check quotas
Verify balance, time windows, and rate limits
- 03Select channel
Select an available upstream by channel configuration and health
- 04Forward request
Transform protocols and proxy to model endpoints
- 05Record usage
Log latency, token consumption, and audit details
Conceptual request flow. Verify execution order, routing policies, and error behavior for your release.
Debug errors without
chasing raw logs.
Track every call with a unique Request ID. Trace model responses, latency, and security flags together in one view.
Read the troubleshooting guide- request_id
- req_example_01
- model
- <MODEL_NAME>
- status
- Request status
- duration
- Elapsed time
- usage
- Tokens & calculated cost
Your infrastructure. Your data.
Self-host on your servers with Docker. Credentials, keys, and usage records stay entirely in your own environment.
Get your AI gateway up
in minutes.
One command to install. Start connecting models and clients right away.
Deploy now