A hosted marketplace and gateway: you buy credits, and OpenRouter resolves a model ID like anthropic/claude-sonnet-5 to one of several upstream providers, falling over automatically when one errors or rate-limits. It charges no markup on tokens — you pay provider list price — and makes its money on a 5.5% fee ($0.80 minimum) on card credit purchases, 5% on crypto. Unlike a self-hosted proxy, it gives you day-one access to models you have no account for, plus per-provider throughput and latency stats that let you route on price or speed. Homepage claims 300T+ tokens routed monthly, which makes it the de facto default in this category.
| Providers supported | 80+ providers, 500+ models |
|---|
| Fallback / routing | Automatic provider failover, price/throughput/latency sort, provider allow/deny lists |
|---|
| Caching | Passes through provider prompt caching; no gateway-owned response cache |
|---|
| Cost tracking | Per-key and per-generation spend, activity dashboard, generation API |
|---|
| Self-hostable | No |
|---|
| Markup / fee | 0% on tokens; 5.5% on credit top-ups; 5% BYOK above $25k/mo |
|---|
Watch out: Single point of failure — an OpenRouter outage takes down every model at once, which is worse than any one provider degrading. Not self-hostable, so prompts transit a third party; data-residency and ZDR guarantees are weaker than going direct. The 5.5% credit fee is dead weight if you already have committed provider spend. Stripe's agreed >$7B acquisition (reported Aug 2026) adds roadmap and pricing uncertainty that a risk-averse enterprise should price in.
$0 markup on tokens; 5.5% fee ($0.80 min) on card credit purchases, 5% crypto; BYOK free up to $25,000/mo of list-price usage ($200,000/mo on Enterprise), then 5%
The open-source default for teams that want a gateway they run themselves. LiteLLM normalizes 100+ providers (OpenAI, Anthropic, Bedrock, Vertex, Azure, vLLM, NIM, Ollama) behind the OpenAI schema, and its proxy server adds virtual API keys, per-key and per-team budgets, spend tracking and logging callbacks. It is free forever self-hosted with no license fee for production; the Enterprise tier adds SSO/SCIM, audit logs, secret-manager integrations, 24/7 SLAs and air-gapped deployment, priced annually against gateway request capacity rather than per token. The SDK is also widely embedded directly in applications without running the proxy at all.
| Providers supported | 100+ providers, 1,800+ model mappings |
|---|
| Fallback / routing | Model fallback lists, retries, load balancing across deployments, latency/cost routing |
|---|
| Caching | Yes — Redis/in-memory exact and semantic caching |
|---|
| Cost tracking | Per-key/team/user spend, budgets, rate limits, DB-backed logs |
|---|
| Self-hostable | Yes — Docker/Helm, air-gapped on Enterprise |
|---|
| Markup / fee | 0% — BYOK only; you pay providers directly |
|---|
Watch out: You own the operations: Postgres, Redis, upgrades and the on-call pager. The project moves very fast and minor releases have broken behaviour and configs in the past; pin versions. Historically Python-based and heavier per-request than Go/Rust gateways, so benchmark tail latency before putting it in front of high-RPS traffic. Observability is spend-first — for prompt-level tracing and evals you will still bolt on Langfuse, Helicone or similar. Enterprise pricing is not public, which makes budgeting awkward.
Open source: $0. Enterprise: custom annual pricing sized to gateway request capacity and support tier (no public list price; third-party reports put entry around $250/mo-equivalent)
· open source
A hosted AI gateway plus observability layer over 250+ models, with an MIT-licensed open-source gateway core you can self-host. Its distinguishing features are config-as-JSON routing (fallback chains, load balance weights, conditional routing on request metadata), a prompt-template registry with versioning, and guardrails that can run on request or response. Pricing is metered on recorded logs rather than tokens: free to 10k logs/month, $49/mo for 100k with $9 per additional 100k up to 3M, then Enterprise for 10M+ with VPC/private-cloud deployment, SSO and HIPAA. Adds roughly 20-40ms of edge latency on the managed offering.
| Providers supported | 250+ models across major providers |
|---|
| Fallback / routing | Fallback chains, weighted load balancing, conditional routing on metadata, retries |
|---|
| Caching | Yes — simple and semantic cache (paid tiers) |
|---|
| Cost tracking | Per-request cost, metadata/user attribution, budgets and RBAC on paid tiers |
|---|
| Self-hostable | Yes — open-source gateway; managed VPC/private cloud on Enterprise |
|---|
| Markup / fee | 0% on tokens; billed per recorded log |
|---|
Watch out: Log-based pricing punishes high-volume, low-value traffic — a chatty agent loop can burn your log quota far faster than your token budget. Free tier's 3-day retention is too short for incident forensics. The open-source gateway is genuinely useful but is a subset: analytics, guardrails and the prompt registry are the hosted product, so self-hosting does not get you feature parity. Adding another managed hop is the wrong choice for latency-critical, single-provider workloads.
Developer $0 (10k logs/mo, 3-day retention); Production $49/mo (100k logs, +$9 per 100k up to 3M); Enterprise custom (10M+ logs); open-source gateway free
· open source
A one-line-change proxy — you swap the provider base URL for a Cloudflare gateway URL — that adds analytics, response caching, rate limiting, retries and model fallback in front of your own keys. The core features cost nothing beyond a Cloudflare account; persistent logs are capped at 100,000 total on Workers Free and 10,000,000 per gateway on Workers Paid. Optional Unified Billing lets you pay Cloudflare for provider inference instead of holding accounts everywhere, at a 5% fee on credit purchases with no markup on the inference itself. Guardrails run on Llama Guard and bill as Workers AI inference; DLP scanning is free.
| Providers supported | Workers AI plus OpenAI, Anthropic, Google Gemini, Bedrock, Replicate, Groq and other major providers |
|---|
| Fallback / routing | Request retry and model fallback; dynamic routing configs |
|---|
| Caching | Yes — response caching at the edge, free |
|---|
| Cost tracking | Request/token/cost analytics per gateway; Logpush to your own sink |
|---|
| Self-hostable | No — runs on Cloudflare's network |
|---|
| Markup / fee | 0% by default (BYOK); 5% on credit purchases if you use Unified Billing |
|---|
Watch out: Not self-hostable and not portable — the gateway URL and its configs are Cloudflare-specific. Log retention and volume are the real constraint: 100k logs on the free plan disappears fast in production. Routing is thinner than LiteLLM or Portkey (no cost/quality-based per-prompt routing, weaker per-team budget and RBAC story), and per-user cost attribution is basic. Wrong choice if you need prompt-level evals or an audit trail you can query like a database.
Core features $0; persistent logs 100k total (Workers Free) / 10M per gateway (Workers Paid, $5/mo); Logpush on Paid with 10M req/mo included then $0.05 per additional million; Unified Billing 5% on credit purchases, 0% inference markup
A credits-based gateway to hundreds of models with an explicit no-markup, no-platform-fee policy on tokens, including when you bring your own provider key. It speaks the AI SDK, OpenAI Chat Completions, OpenAI Responses and Anthropic Messages APIs, retries to other providers on failure, and exposes budgets plus per-generation cost and latency. Where it differs from OpenRouter is billing shape and add-ons: tokens are at list price with no top-up percentage, but optional features are metered — Custom Reporting at $0.075 per 1,000 tag/user-ID writes and $5 per 1,000 report queries, team-wide provider allowlist and team-wide ZDR at $0.10 per 1,000 requests each. Every team gets monthly free credits on a subset of models until the first credit purchase.
| Providers supported | Hundreds of models across major providers; BYOK for any catalog provider |
|---|
| Fallback / routing | Automatic retry to alternate providers, routing rules, provider preference/only filters |
|---|
| Caching | Provider prompt caching passthrough; no gateway-owned semantic cache |
|---|
| Cost tracking | Credit balance, per-generation cost/latency/tokens, budgets, custom reporting by tag/user (metered) |
|---|
| Self-hostable | No |
|---|
| Markup / fee | 0% on tokens and 0% BYOK fee; payment processing fees on top; add-on features metered separately |
|---|
Watch out: You are billed through Vercel and dependent on Vercel's availability; payment processing fees apply unless you are Enterprise and invoiced. The zero-markup headline hides metered add-ons — attributing spend per end user via Custom Reporting has a real per-write cost, and team-wide ZDR and provider allowlist are billed per request and gated to Pro/Enterprise. Not self-hostable, no semantic caching of its own, and the free tier only covers a model subset. Wrong choice if you are not otherwise in the Vercel ecosystem and want a gateway you can run in your own VPC.
0% markup and no platform fee on tokens (BYOK included); Custom Reporting $0.075/1,000 writes and $5/1,000 queries; team-wide provider allowlist $0.10/1,000 requests; team-wide ZDR $0.10/1,000 requests; volume discounts on Enterprise
Helicone started as a one-line observability proxy and now ships two things: the hosted logging/analytics platform, and the Helicone AI Gateway, an MIT-licensed single-binary router written in Rust that handles failover, load balancing, rate limiting and caching across 100+ models. Both are fully open source with no commercial restriction, so the self-host path is real rather than a crippled community edition. Hosted pricing is Hobby free to 10,000 requests/month with 7-day retention, Pro $79/mo with unlimited seats and usage-based overage, Team $799/mo for SOC-2/HIPAA and 3-month retention, Enterprise for on-prem. The gateway can be run standalone via Docker or npx with sub-10ms added latency.
| Providers supported | 100+ models across major providers via the AI Gateway |
|---|
| Fallback / routing | Failover, load balancing across providers, rate limiting |
|---|
| Caching | Yes — response caching in the gateway |
|---|
| Cost tracking | Per-request cost, user/session attribution, HQL query language, alerts and reports (Pro+) |
|---|
| Self-hostable | Yes — MIT single binary via Docker/npx/K8s; on-prem platform on Enterprise |
|---|
| Markup / fee | 0% on tokens — BYOK; billed on requests/storage |
|---|
Watch out: Retention is the hidden cost — 7 days free and 1 month on Pro is short if you need to investigate a regression from last quarter, and storage/token overages on Pro are usage-based rather than flat. The Team tier's jump from $79 to $799/mo is steep for the compliance features it unlocks. The AI Gateway is newer and less battle-tested than LiteLLM, with thinner enterprise governance (budgets, virtual key hierarchies, SSO) unless you buy the platform. Self-hosting the full platform including ClickHouse is meaningfully more work than running the gateway binary alone.
Hobby $0 (10k requests/mo, 1GB storage, 7-day retention); Pro $79/mo (unlimited seats, 10k free requests then usage-based, 1-month retention); Team $799/mo (SOC-2/HIPAA, 3-month retention); Enterprise custom
· open source
Rather than a new product, this is a family of AI plugins on the existing Kong Gateway data plane — ai-proxy, ai-proxy-advanced, semantic caching, semantic prompt guard, PII sanitization, token rate limiting — so LLM traffic is governed by the same Kong policies, auth and observability as the rest of your APIs. It has expanded to cover MCP servers and agent-to-agent traffic, with L7 telemetry on payloads, latency and token spend, and showback/chargeback quotas per user, model or time window. Konnect (the managed control plane) has a 30-day full-feature free trial; the Plus tier is charged per gateway per month and includes 1M API requests with additional analytics at $200/mo per extra million, capped at 5 unique LLM models. Enterprise removes the model cap.
| Providers supported | OpenAI, Anthropic, Azure, Bedrock, Vertex, Mistral, Cohere, Llama/self-hosted and other OpenAI-compatible endpoints |
|---|
| Fallback / routing | Semantic routing, load balancing, retries and multi-LLM failover (ai-proxy-advanced) |
|---|
| Caching | Yes — semantic caching (Enterprise plugin, needs a vector store) |
|---|
| Cost tracking | Token counting, per-user/model quotas, showback/chargeback, L7 analytics |
|---|
| Self-hostable | Yes — self-managed data planes; hybrid or fully on-prem |
|---|
| Markup / fee | 0% on tokens; licensed per gateway/requests |
|---|
Watch out: The best plugins — semantic caching, prompt guard, advanced load balancing — are Enterprise, and semantic caching additionally requires you to stand up and operate a vector store. Pricing is opaque: the Plus per-gateway dollar figure is not published, and the 5-model cap on Plus is a real constraint for anyone experimenting across labs. Operationally it is a full API gateway, so for a small team that just wants failover across two providers it is dramatically more machinery than the problem warrants. Developer ergonomics lag purpose-built gateways: no prompt registry, no eval workflow, weaker per-request cost UX.
Konnect free 30-day trial; Plus charged per gateway/month (dollar amount not published on the pricing page) with 1M requests/mo included and $200/mo per additional 1M; Enterprise custom annual. Open-source Kong Gateway is $0 with a subset of AI plugins
· open source
A European alternative to OpenRouter: one endpoint to 600+ models across 20+ providers, with smart routing, caching, failover, RBAC and MCP support, and EU data residency available on every plan including the free tier. Its pricing is deliberately simpler and more expensive than the credit-fee model — a flat 5% on model cost (a $10/1M-token model bills at $10.50), no per-seat charge and no minimum spend — and BYOK is supported on pay-as-you-go. The free tier gives access to free models at 200 requests/day. Enterprise adds SSO, full RBAC, audit logs, guardrails, PII detection and custom SLAs.
| Providers supported | 20+ providers, 600+ models |
|---|
| Fallback / routing | Smart routing, automatic failover across providers |
|---|
| Cost tracking | Spend tracking per key/user, dashboards |
|---|
| Caching | Yes — included on all plans |
|---|
| Self-hostable | No |
|---|
| Markup / fee | 5% markup on token cost (not on top-ups); BYOK supported |
|---|
Watch out: The 5% applies to every token you spend, not just to credit top-ups, so at scale it costs more than OpenRouter's or Cloudflare's top-up fee model — run the arithmetic against your monthly spend before committing. It is a five-person seed-stage company with roughly $3M raised, so vendor-continuity risk is materially higher than the Cloudflare/Vercel/Kong options; keep a direct-to-provider path. Not self-hostable, no open-source core, and the free tier's 200 requests/day is a demo allowance rather than a development budget.
Free $0 (free models, 200 requests/day); Pay-as-you-go 5% markup on model cost, no seat fees or minimums; Enterprise custom
An open-source AI gateway written in Go, positioned explicitly as the low-overhead alternative to Python proxies: published benchmarks claim roughly 11 microseconds of added overhead per request at 5,000 RPS, about 54x lower P99 than a Python-based proxy on the same hardware, and 68% less memory. It exposes one OpenAI-compatible interface over 20+ providers (OpenAI, Anthropic, Bedrock, Vertex, Azure, Mistral, Groq, Cohere, Ollama, vLLM) plus native MCP support, adaptive load balancing, semantic caching and automatic failover, and drops into existing SDKs by changing only the base URL. The OSS build is free forever under Apache 2.0; Enterprise adds clustering/HA, guardrails, SAML/OIDC, vault integrations and SLA support at custom pricing.
| Providers supported | 20+ providers including OpenAI, Anthropic, Bedrock, Vertex, Azure, Groq, Ollama, vLLM |
|---|
| Fallback / routing | Automatic failover, routing rules, adaptive load balancing (adaptive LB on Enterprise) |
|---|
| Caching | Yes — semantic caching in OSS |
|---|
| Cost tracking | Built-in observability and token/cost metrics; deeper analytics via Maxim platform |
|---|
| Self-hostable | Yes — this is the primary deployment mode |
|---|
| Markup / fee | 0% — BYOK only |
|---|
Watch out: Far smaller ecosystem and community than LiteLLM, with 20+ providers rather than 100+, so an unusual provider may need you to write the integration. Benchmarks are vendor-published; verify on your own hardware and traffic shape. Clustering/HA, guardrails and the MCP gateway sit behind Enterprise with no published price, so the free tier is single-node in practice. Governance features (virtual key hierarchies, team budgets, audit) are less mature than LiteLLM Enterprise or Portkey, and the gravitational pull toward Maxim's eval platform is real.
OSS $0 forever (Docker, Kubernetes, Go binary); Enterprise custom pricing with a 14-day free trial — no public list price
· open source
The open-source, vendor-neutral option for platform teams already running Kubernetes and Envoy. Built on the CNCF Envoy Gateway project and the Kubernetes Gateway API Inference Extension, it reached v1.0 in June 2026 with 16 AI providers, an MCP gateway, multimodal support, token-based rate limiting, per-request credentials, OpenTelemetry GenAI tracing, stream idle timeouts and HTTP CONNECT egress. It routes to both hosted providers and self-hosted inference clusters (vLLM and friends), with inference-aware endpoint selection and load balancing. Configuration is CRDs, not a dashboard.
| Providers supported | 16 AI providers at v1.0, plus any self-hosted OpenAI-compatible endpoint |
|---|
| Fallback / routing | Unified API routing, inference-aware endpoint selection and load balancing, retries |
|---|
| Caching | No first-class semantic cache; use Envoy filters or an external cache |
|---|
| Cost tracking | Token counting, token-based rate limits, OpenTelemetry GenAI traces — no built-in cost dashboard |
|---|
| Self-hostable | Yes — Kubernetes only, this is the only deployment mode |
|---|
| Markup / fee | 0% — BYOK only |
|---|
Watch out: Kubernetes-only and CRD-driven — there is no dashboard, no hosted option and no five-minute path for an application team. No built-in cost analytics or spend attribution UI; you are expected to pipe OTel traces somewhere and build that yourself. Provider coverage (16) is a fraction of LiteLLM's, and semantic caching is not a first-class feature. v1.0 is recent, so operational patterns are still maturing. Wrong choice for a small team or anyone who wants a product rather than infrastructure.
$0 — Apache 2.0 open source, no commercial tier from the project itself; commercial support available from Tetrate at unpublished pricing
· open source
An enterprise-oriented gateway that unifies model access, routing, guardrails and cost controls, and is unusual in this category for publishing per-request list pricing rather than hiding everything behind sales. Developer is free at 50k requests/month with 3 users; Pro is $499/mo for 1M requests and 10 users with advanced routing, semantic caching and a standard SLA; Pro Plus is $2,999/mo for 1M requests and 25 users with dedicated onboarding and stronger data controls; Enterprise covers 10M+ requests with on-prem, VPC and air-gapped deployment. Overage runs about $499/mo per additional 2M requests. It ships as part of a broader ML platform that also does model deployment and agent infrastructure.
| Providers supported | Major hosted providers plus self-hosted/OpenAI-compatible endpoints (1,000+ model catalog claimed) |
|---|
| Fallback / routing | Multiple routing strategies, load balancing, fallbacks, retries |
|---|
| Caching | Yes — semantic caching on Pro and above |
|---|
| Cost tracking | Per-team/user cost attribution, budgets, rate limits, observability with custom retention |
|---|
| Self-hostable | Yes — VPC, on-prem and air-gapped on Enterprise |
|---|
| Markup / fee | 0% on tokens; licensed per request volume and seats |
|---|
Watch out: Seat caps bite before request caps for larger orgs — 10 users on a $499/mo plan is restrictive, and the jump to Pro Plus is 6x the price for the same 1M requests. Closed source, so there is no escape hatch if you outgrow the vendor. It is a broader ML platform, and buying the gateway can pull you toward the rest of the stack. Smaller community and less third-party integration coverage than LiteLLM or Portkey; much of the enterprise governance story still requires a sales conversation.
Developer $0 (50k requests/mo, 3 users); Pro $499/mo (1M requests, 10 users); Pro Plus $2,999/mo (1M requests, 25 users); Enterprise custom (10M+ requests); overage ~$499/mo per additional 2M requests
Not a gateway but a true router: it inspects each prompt and picks the model most likely to answer it well at the lowest cost, rather than routing by static config. Pricing is $0.05 per million tokens routed on pay-as-you-go, with volume discounts, SSO/SAML, org-wide analytics and privacy-preserving deployment on Enterprise; it is SOC-2 and ISO 27001 compliant. The company claims 20-40% cost savings without quality degradation, and routing adds roughly 100-150ms of latency per request. Recent work targets coding agents specifically, including Claude Code harnesses, with an early-access coding-agent router.
| Providers supported | Multi-provider across major labs; model set is the routing candidate pool, not a full catalog |
|---|
| Fallback / routing | Per-prompt quality/cost routing — this is the entire product |
|---|
| Caching | No — not a caching layer |
|---|
| Cost tracking | Savings dashboard and org-wide analytics (Enterprise) |
|---|
| Self-hostable | Privacy-preserving deployment options on Enterprise; not generally self-hostable |
|---|
| Markup / fee | $0.05 per 1M tokens routed, on top of provider token cost |
|---|
Watch out: 100-150ms of added latency per request is disqualifying for streaming-first UX and multiplies badly inside agent loops that make many sequential calls. Routing makes your output non-deterministic across models, which breaks prompt tuning, eval baselines and anything sensitive to a specific model's formatting behaviour. The savings claim is workload-dependent — if your traffic is uniformly hard, the router picks the expensive model every time and you have paid a fee for nothing. It is a router, not a gateway: no caching, no key management, no full model catalog, so you will still run something else alongside it.
$0.05 per million tokens routed (pay-as-you-go); Enterprise custom with volume discounts
An AI gateway written entirely in Rust, offered both as a self-hosted open-source binary and a managed service, with a unified OpenAI-compatible API over 250+ models. It emphasizes agent observability alongside routing: tracing and analytics on agent runs, dynamic routing on cost and quality signals, real-time cost control, and integrations with LangChain and CrewAI. The Rust core targets the same low-overhead niche as Bifrost and Helicone's gateway rather than the feature breadth of LiteLLM.
| Providers supported | 250+ models across major providers |
|---|
| Fallback / routing | Dynamic routing on cost and quality, fallbacks, load balancing |
|---|
| Caching | Yes — caching supported in the gateway |
|---|
| Cost tracking | Real-time cost control, per-run tracing and analytics |
|---|
| Self-hostable | Yes — open-source Rust gateway |
|---|
| Markup / fee | unknown — verify current fee structure with the vendor |
|---|
Watch out: Pricing could not be verified from the official site, so treat any cost planning as unconfirmed until you get a quote — that alone is a reason not to commit without a sales conversation. Much smaller community and integration surface than LiteLLM or Portkey, and less independent operational reporting at scale. The open-source/managed feature split is not clearly documented publicly, so confirm which capabilities survive self-hosting before designing around them. Low confidence overall on this entry.
unknown — the pricing page advertises a free start with paid scaling but published tier figures could not be verified from the official site (403 on automated fetch as of 2026-09-05)
· open source
A smaller open-source competitor to OpenRouter that offers both a hosted marketplace and a genuinely self-hostable AGPLv3 build with the same feature set. Hosted: 200+ models across 40+ providers, no token markup, a 5% platform fee on credit purchases only, and 0% if you bring your own provider keys. The free plan gives access to the full 200+ model catalog with three rate-limited free models and 30-day data retention; full retention costs $0.01 per 1M tokens stored. International cards add 1.5%. Enterprise adds volume discounts, a 99.9% uptime SLA, custom routing rules, SSO/SAML and audit logs.
| Providers supported | 40+ providers, 200+ models |
|---|
| Fallback / routing | Automatic failover; custom routing rules on Enterprise |
|---|
| Caching | Yes — built-in caching |
|---|
| Cost tracking | Cost analytics per key and model; audit logs on Enterprise |
|---|
| Self-hostable | Yes — free under AGPLv3 |
|---|
| Markup / fee | 0% on tokens; 5% on credit purchases; 0% with BYOK |
|---|
Watch out: AGPLv3 is a genuine constraint — legal review will ask questions if you modify the gateway and expose it as a service. Model catalog (200+) and provider count (40+) are well below OpenRouter's, so exotic or day-one models may be missing. It is a small project with correspondingly small operational track record; do not put it on the critical path without your own failover. The extra 1.5% for international cards and $0.01/1M-token retention fee are easy to miss when comparing headline rates.
Free $0 (200+ models, 3 rate-limited free models, 30-day retention); Paid pay-per-token + 5% fee on credit purchases, 0% with BYOK; storage $0.01 per 1M tokens for full retention; +1.5% for international cards; Enterprise custom. Self-host free under AGPLv3
· open source
A gateway that exists to feed an eval platform. The proxy provides provider routing, caching, cost visibility and request logging behind one OpenAI-compatible endpoint, and its value is that every request lands in Braintrust Logs where it can be added to datasets, scored with scorers and classifiers, compared in experiments, monitored with online scoring and wired into CI/CD checks. The proxy itself is open source (braintrust-proxy on GitHub) though the platform is closed. Pricing shifted in March 2026: the Starter plan replaced the free tier with 1 GB of processed data per month, 10,000 scores, unlimited users and unlimited projects; Pro is $249/mo with usage-based data billing above the included allowance.
| Providers supported | Major providers (OpenAI, Anthropic, Google, Bedrock, Azure and OpenAI-compatible endpoints) |
|---|
| Fallback / routing | Provider routing and failover; not a cost/quality per-prompt router |
|---|
| Caching | Yes — built into the proxy |
|---|
| Cost tracking | Cost visibility per request, tied to logs, datasets and experiments |
|---|
| Self-hostable | Proxy yes (open source, deployable to your own edge); platform on Enterprise only |
|---|
| Markup / fee | 0% on tokens; billed on processed data volume and scores |
|---|
Watch out: Data-volume pricing is the trap: 1 GB of processed data per month is easy to blow through with long contexts, and costs scale with payload size rather than requests, so a few large-context workloads can dominate the bill. It is a weak choice as a pure gateway — governance features like virtual key hierarchies, per-team budgets and enterprise SSO are thinner than Portkey or LiteLLM Enterprise. The platform is closed source even though the proxy is not, so self-hosting the proxy alone loses the entire reason to use it. Wrong choice if you already own your eval tooling.
Starter $0 (1 GB processed data/mo, 10,000 scores, unlimited users and projects); Pro $249/mo plus usage-based data billing; Enterprise custom
· open source
A governance layer bolted onto Databricks Model Serving rather than a standalone gateway. It centralizes inference routing with usage tracking, rate limiting and policy enforcement, supports external models (OpenAI, Anthropic and others registered as External Model endpoints), does fallback routing across providers when one is down or throttled, and captures request/response payloads into Unity Catalog tables so governance, lineage and access control are the same as for your data. Billing is DBU-based on the Databricks meter — Mosaic AI at roughly $0.07/DBU, with foundation-model serving priced in DBU per 1M tokens and GPU serving in DBU-hours — rather than a separate gateway SKU.
| Providers supported | Databricks-hosted foundation models plus External Models (OpenAI, Anthropic, Cohere, Bedrock and others) |
|---|
| Fallback / routing | Fallback chains across providers, traffic splitting across served entities |
|---|
| Caching | No first-class semantic cache in the gateway |
|---|
| Cost tracking | Usage tracking and payload logging into Unity Catalog tables; system tables for chargeback |
|---|
| Self-hostable | No — runs in your Databricks workspace/cloud account only |
|---|
| Markup / fee | DBU consumption on top of provider token cost; no explicit percentage markup |
|---|
Watch out: Only makes sense if you are already a Databricks shop — buying Databricks to get a gateway is absurd. DBU pricing is notoriously hard to forecast for LLM traffic, and the gateway cost is entangled with serving cost rather than itemized. No semantic caching, a narrower provider catalog than a marketplace gateway, and no self-hosted or portable deployment. Iteration speed on model availability lags the dedicated gateways, so day-one access to a new frontier model is not guaranteed.
DBU-metered on top of your Databricks contract: Mosaic AI ~$0.07/DBU, Model Training ~$0.65/DBU, GPU Model Serving ~10.48-628 DBU/hr; Foundation Model APIs priced in DBU per 1M tokens. No standalone gateway price
Microsoft's answer for enterprises that already run Azure API Management: a set of GenAI policies applied to LLM backends, covering token-limit policies keyed on subscription, IP or a policy expression (with prompt tokens pre-calculated at the gateway so over-limit requests never hit the backend), token quotas, semantic caching via llm-semantic-cache-store and llm-semantic-cache-lookup against Azure Managed Redis or another RediSearch-compatible cache, Azure AI Content Safety integration, and fallback to a secondary model. Coverage now extends beyond Azure OpenAI to Anthropic and Vertex AI traffic. A dedicated AI Gateway tier of API Management is in public preview; the GenAI policies themselves are usable on the standard tiers today.
| Providers supported | Azure OpenAI plus Anthropic, Vertex AI and other OpenAI-compatible backends you configure |
|---|
| Fallback / routing | Backend pools with load balancing, circuit breaker, fallback to a secondary model |
|---|
| Caching | Yes — semantic caching, but requires you to run Azure Managed Redis or equivalent |
|---|
| Cost tracking | Token metrics emitted to Application Insights/Event Hubs; token quotas per counter key |
|---|
| Self-hostable | Self-hosted gateway containers available for hybrid/on-prem data planes |
|---|
| Markup / fee | 0% on tokens; you pay APIM capacity plus cache infrastructure |
|---|
Watch out: Expensive floor — roughly $700/mo for a Standard v2 unit before you serve a single token, plus a Redis cluster if you want semantic caching. Configuration is XML policy documents, not a friendly console, and the developer experience is nothing like a purpose-built LLM gateway: no model catalog, no prompt registry, no per-request cost UI out of the box. The dedicated AI Gateway tier is still public preview, so do not build a production commitment on its specific behaviour yet. Effectively Azure-only; wrong choice for a multi-cloud or non-Azure team.
Billed as API Management capacity: Standard v2 ~$0.9589/hour in East US (~$700/mo) including 50M API requests; semantic caching adds an external RediSearch-compatible cache from ~$0.016 to ~$0.216/hour. Dedicated AI Gateway tier is in public preview
A fully Apache-2.0 API gateway that added LLM capability through plugins rather than a separate product. The ai-proxy plugin transforms requests into the target provider's format; ai-proxy-multi extends it with load balancing, retries, fallbacks and health checks across OpenAI, DeepSeek, Azure, AIMLAPI and any OpenAI-compatible endpoint. It logs LLM-specific fields into the access log — token usage, model, time to first token — so LLM traffic shows up in the same observability pipeline as your REST traffic, and it also fronts MCP server traffic. Everything is free; there is no enterprise plugin tier gating the AI features the way Kong has one.
| Providers supported | OpenAI, DeepSeek, Azure, AIMLAPI and any OpenAI-compatible endpoint; no curated model catalog |
|---|
| Fallback / routing | Load balancing, retries, fallbacks and health checks via ai-proxy-multi |
|---|
| Caching | No dedicated semantic cache plugin; use APISIX's general caching or an external layer |
|---|
| Cost tracking | Token usage and TTFT in access logs and Prometheus metrics; no cost dashboard |
|---|
| Self-hostable | Yes — this is the only deployment mode |
|---|
| Markup / fee | 0% — BYOK only |
|---|
Watch out: Native provider coverage is narrow compared to LiteLLM — anything not OpenAI-compatible needs work. No cost dashboards, no per-team budgets, no prompt registry and no semantic caching plugin; you get logs and metrics and build the rest. Configuration is etcd-backed routes and plugin config, so the developer onboarding path is much longer than a base-URL swap. Community around the AI plugins specifically is smaller than around the core gateway. Wrong choice if you are not already an APISIX shop.
$0 — Apache 2.0, all AI plugins included; commercial support and the API7 Enterprise control plane priced separately (not published)
· open source
Martian built one of the first commercial per-prompt LLM routers, using its 'model mapping' interpretability technique to predict which model would answer a given prompt best, and raised at a reported $1.3B valuation in 2026. As of September 2026 the public posture has shifted decisively toward interpretability research — ARES, Code Review Bench, K-Steering, feature geometry — and the company describes commercializing that research through future products. The historical model-router property (route.withmartian.com) did not resolve over TLS on 2026-09-05, and the router is not presented as a product on the main site. Treat availability as unverified.
| Providers supported | unknown — historically major frontier labs as routing candidates |
|---|
| Fallback / routing | Per-prompt model routing plus rerouting on provider outage (as historically marketed) |
|---|
| Caching | unknown |
|---|
| Cost tracking | unknown |
|---|
| Self-hostable | No |
|---|
| Markup / fee | unknown |
|---|
Watch out: Do not put this on a production path without direct confirmation from the vendor that the router is still sold and supported. The router URL failed to establish a TLS connection during verification and the product is absent from the company's current public messaging, which is the classic signature of a quietly sunset product. Third-party comparison articles that still list it are likely stale. Everything in this entry is low confidence.
unknown — no public pricing page reachable for the model router as of 2026-09-05
· unknown
Unify offered a benchmark-driven router that continuously measured providers on live prompts and steered traffic to the best cost/latency/quality candidate, and was widely recommended in 2024-2025 comparison articles. As of 2026-09-05 unify.ai serves only the line 'Bridging symbolic reasoning and continual learning. More coming soon.' — the router product and its pricing are gone. It is included here specifically because a large amount of stale SEO content still recommends it, and practitioners will find those recommendations before they find the actual homepage.
| Providers supported | Historically many hosted inference providers; no longer offered |
|---|
| Fallback / routing | Benchmark-driven quality/cost/latency routing (discontinued) |
|---|
| Caching | n/a |
|---|
| Cost tracking | n/a |
|---|
| Self-hostable | No |
|---|
| Markup / fee | n/a |
|---|
Watch out: The product is gone. Any 2026 blog post or comparison table still recommending Unify as an LLM router is stale; verify against unify.ai before acting on it. If you were running on Unify, you need a migration plan to OpenRouter, LiteLLM or Not Diamond depending on whether you valued the marketplace, the self-hosting or the per-prompt routing.
unknown — no pricing page; the router product is no longer offered
· discontinued