Together runs a large serverless catalog (200+ text, vision, image, video and audio models) alongside dedicated endpoints and rentable GPU clusters, so you can start per-token and move the same model onto reserved hardware without changing the API. Pricing as of September 2026: DeepSeek V4 Flash 0731 $0.14/$0.28 per 1M, DeepSeek V4 Pro $1.32/$3.96, Qwen3.8 Flash $0.15/$0.47, Llama 3.3 70B $1.04/$1.04, Kimi K3 $3.00/$15.00, GLM-5.3-Flash $0.15/$0.50. Dedicated inference is $5.49/GPU/hr for H100 (promo $3.99) and $8.99 for B200; GPU clusters run $3.99/hr H100 on-demand down to $3.19/hr reserved at 181+ days. The differentiator against Fireworks is the cluster business — Together will sell you bare training capacity, not just inference.
| Output speed, gpt-oss-120b | ~94 tok/s (Artificial Analysis) |
|---|
| Models hosted | 200+ |
|---|
| Dedicated endpoints | Yes — dedicated model + container inference, provisioned throughput |
|---|
| Llama/DeepSeek/Qwen-class price | DeepSeek V4 Flash $0.14/$0.28 per 1M; Qwen3.8 Flash $0.15/$0.47 |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | Signup credits; one $0.00 model |
|---|
Watch out: Throughput is unremarkable — Artificial Analysis clocks Together at ~94 tok/s on gpt-oss-120b, roughly 20x slower than Cerebras and slower than DeepInfra's turbo tier — so it is the wrong choice for latency-sensitive voice or tight agent loops. Serverless p99 latency varies with shared-capacity load, and the headline dedicated H100 rate ($5.49/hr) is well above DeepInfra ($2.20/hr) for the same silicon.
DeepSeek V4 Flash $0.14/$0.28 per 1M; Llama 3.3 70B $1.04/$1.04; Kimi K3 $3.00/$15.00; dedicated H100 $5.49/GPU/hr; clusters from $3.19/GPU/hr reserved
Fireworks pairs a fast-moving serverless catalog (hundreds of LLMs, VLMs, embedding and reranker models, with GLM 5.3 Flash, DeepSeek-V4-Pro at 1M context, Qwen3.8 Flash Next and Kimi K3 all live) with on-demand GPU deployments and a managed fine-tuning product. Serverless is tiered — Standard, Priority and Fast — so you buy latency rather than getting one shared queue; Artificial Analysis measures 273 tok/s on DeepSeek V4 Flash and 521 tok/s on Nemotron 3.5 Lightning at $0.04/1M. On-demand GPUs rose on 1 September 2026 to $8.00/hr for H100 and H200, $13.00 for B200, $15.00 for B300 and $20.00 for GB300. Managed training is priced per token — LoRA SFT from $0.50/1M on models up to 16B, $3.00/1M on 16-80B. It has become the largest pure-play inference vendor by revenue (>$1B ARR, $17.5B valuation as of July 2026).
| Output speed, gpt-oss-120b | Not published by Artificial Analysis; 273 tok/s on DeepSeek V4 Flash |
|---|
| Models hosted | Hundreds (LLM, VLM, embedding, reranker); no official count |
|---|
| Dedicated endpoints | Yes — on-demand and reserved GPU deployments |
|---|
| Llama/DeepSeek/Qwen-class price | DeepSeek V4 Flash ~$0.11/1M blended; Nemotron 3.5 Lightning $0.04/1M |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | $1 credit |
|---|
Watch out: The GPU price list jumped sharply on 1 September 2026 (H100 $7 to $8, B200 $10 to $13, B300 $12 to $15), so on-demand GPU economics are worse than DeepInfra, Lambda or Runpod for the same hardware and you should assume further moves. Per-model serverless rates are not published on the marketing pricing page — you have to dig into docs or the model cards — which makes cost modelling harder than at Together or DeepInfra. The three-tier serverless model also means the advertised low price is the slow tier.
DeepSeek V4 Flash ~$0.11/1M blended, GLM-5.3 ~$1.35/1M, Kimi K3 ~$4.22/1M (Artificial Analysis); on-demand H100/H200 $8.00/hr, B200 $13.00/hr, B300 $15.00/hr, GB300 $20.00/hr; LoRA SFT from $0.50/1M tokens
Baseten's core product is Truss-based deployment of arbitrary models onto dedicated autoscaling GPUs, with a per-token Model API layered on top for the popular open weights so you can prototype without provisioning anything. Model API rates as of September 2026: GPT OSS 120B $0.10/$0.50 per 1M, DeepSeek V4 Flash $0.13/$0.26, GLM-5.3-Flash $0.15/$0.50, GLM-5.3 $1.40/$4.40, DeepSeek V4 Pro $1.74/$3.48, Kimi K3 $3.00/$15.00. Dedicated compute is billed per minute: H100 $0.10833/min ($6.50/hr), B200 $0.16633/min ($9.98/hr), A100 $0.06667/min ($4.00/hr), L4 $0.01414/min. Unlike Together or Fireworks it is genuinely a platform for your model, not just a catalog, and it sells self-hosted/VPC deployment for regulated buyers. It raised a $1.5B Series F in 2026 at roughly $13B on ~$600M ARR.
| Output speed, gpt-oss-120b | 267 tok/s (Artificial Analysis) |
|---|
| Models hosted | ~22 on the Model API; unlimited via custom deployment |
|---|
| Dedicated endpoints | Yes — the primary product, incl. self-hosted/VPC |
|---|
| Llama/DeepSeek/Qwen-class price | DeepSeek V4 Flash $0.13/$0.26 per 1M; GPT OSS 120B $0.10/$0.50 |
|---|
| OpenAI-compatible | Yes (Model APIs); custom endpoints are whatever you build |
|---|
| Free tier | Signup credits |
|---|
Watch out: Dedicated GPU pricing is at the top of the market — $6.50/hr for an H100 is roughly 3x DeepInfra's $2.20/hr and above Runpod's $2.69-4.79/hr — so you are paying a large premium for the control plane. The per-token Model API catalog is much narrower than Together's or Novita's (about 22 models tracked), and throughput is mid-pack at ~267 tok/s on gpt-oss-120b. Wrong choice if you just want the cheapest tokens for a standard open model.
GPT OSS 120B $0.10/$0.50 per 1M; DeepSeek V4 Flash $0.13/$0.26; Kimi K3 $3.00/$15.00; dedicated H100 $0.10833/min (~$6.50/hr), B200 $0.16633/min (~$9.98/hr), A100 ~$4.00/hr
GroqCloud serves a deliberately small set of open models on custom LPU hardware at very low latency: GPT OSS 120B at 500 tok/s and $0.15/$0.60 per 1M, GPT OSS 20B at 1,000 tok/s and $0.075/$0.30, Llama 3.1 8B at 560 tok/s, Llama 3.3 70B at 280 tok/s, plus preview Qwen3.6/3.8-27B and the Groq Compound agentic system with built-in web search and code execution at ~450 tok/s. The corporate situation matters as much as the tech: in December 2025 NVIDIA paid roughly $20B to hire most of Groq's engineering team and license the LPU dataflow technology (now shipping as NVIDIA's Groq 3 LPX), and the remaining company raised $650M and then $350M at a $3.5B valuation — about half its pre-deal worth — to operate as a neocloud running both LPUs and NVIDIA GPUs.
| Output speed, gpt-oss-120b | 475 tok/s (Artificial Analysis); 500 tok/s claimed |
|---|
| Models hosted | ~8 production + preview |
|---|
| Dedicated endpoints | Enterprise/dedicated LPU capacity by contract only |
|---|
| Llama/DeepSeek/Qwen-class price | Llama 3.3 70B ~$0.59/$0.79 per 1M; no DeepSeek/Kimi flagship |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | Yes, rate-limited |
|---|
Watch out: Serious platform risk: NVIDIA hired away most of the engineering team and licensed the core IP, and the company re-raised at roughly half its former valuation. The catalog has narrowed to about eight models with no current DeepSeek, Kimi or GLM flagship, so if your product depends on the newest open weights this is not a viable primary. Long-context and very large MoE models are not served, official per-model pricing is hard to find on the public site, and Artificial Analysis measures 475 tok/s on gpt-oss-120b — fast, but now well behind Cerebras.
GPT OSS 120B $0.15/$0.60 per 1M; GPT OSS 20B $0.075/$0.30; Llama 3.1 8B ~$0.05/1M blended; batch API roughly 25% of on-demand rate
Cerebras runs models on wafer-scale CS-series systems and posts throughput no GPU or LPU provider matches: 1,849 tok/s on gpt-oss-120b (high reasoning) and 1,779 tok/s on Gemma 4 31B, versus 150-300 tok/s from mainstream GPU providers. The public pay-per-token rate card as of mid-2026 lists gpt-oss-120b at $0.35/$0.75 per 1M, Gemma 4 31B at $0.99/$1.49 and ZAI-GLM-4.7 at $2.25/$2.75, though only gpt-oss-120b is genuinely production-classified — the others are preview or slated for deprecation. Llama, Qwen3, Mistral and DeepSeek families are available only through dedicated endpoints on custom reserved-capacity pricing, not the public card. Entry is $5 in trial credits, then a self-serve Developer tier from $10.
| Output speed, gpt-oss-120b | 1,849 tok/s (Artificial Analysis) — fastest measured |
|---|
| Models hosted | ~4 on the public rate card |
|---|
| Dedicated endpoints | Yes — reserved capacity, custom pricing, required for Llama/Qwen/DeepSeek |
|---|
| Llama/DeepSeek/Qwen-class price | Not on public card — dedicated endpoints only |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | $5 credits |
|---|
Watch out: The catalog is effectively four models, only one of which is production-grade on the public rate card; everything else (Llama, Qwen3, DeepSeek, Mistral) requires a reserved-capacity contract. That makes it unusable as a general-purpose backend and dangerous to build a product around, because a model you depend on can move to preview or deprecation. It is also not the cheapest — $0.35/$0.75 on gpt-oss-120b is several times DeepInfra's or CoreWeave's blended rate — so you are explicitly buying speed, and capacity has historically been the constraint.
gpt-oss-120b $0.35/$0.75 per 1M; Gemma 4 31B $0.99/$1.49; ZAI-GLM-4.7 $2.25/$2.75; Developer tier from $10; Enterprise by quote
SambaNova serves open models on its own RDU (reconfigurable dataflow unit) silicon, landing second to Cerebras on throughput at 709 tok/s on gpt-oss-120b while charging less than Cerebras for it. The published cloud rate card as of September 2026 covers gpt-oss-120b $0.22/$0.59, Meta-Llama-3.3-70B $0.60/$1.20, gemma-4-31B-it $0.38/$1.15, MiniMax-M2.7 $0.60/$2.40 with $0.06 cached input, MiniMax-M3 $0.60/$2.40, and DeepSeek-V3.1 and V3.2 both at $3.00/$4.50. The company's 2026 pitch is air-cooled inference hardware that drops into existing data centres, so the on-prem/appliance path is a real product rather than a slide.
| Output speed, gpt-oss-120b | 709 tok/s (Artificial Analysis) |
|---|
| Models hosted | ~7 on the public cloud rate card |
|---|
| Dedicated endpoints | Yes — enterprise and on-prem RDU appliances, pricing by quote |
|---|
| Llama/DeepSeek/Qwen-class price | Llama 3.3 70B $0.60/$1.20; DeepSeek V3.2 $3.00/$4.50 per 1M |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | Yes, rate-limited developer tier |
|---|
Watch out: Small catalog (about seven models on the public card) and it is visibly behind on model freshness — DeepSeek is still at V3.1/V3.2 while Together, DeepInfra and Fireworks all serve V4, and those V3.x endpoints are priced at $3.00/$4.50 per 1M, far above what V4 Flash costs elsewhere. No published dedicated-endpoint rate card or throughput SLA on the pricing page. Wrong choice if you need the current frontier open weights or a broad model menu.
gpt-oss-120b $0.22/$0.59 per 1M; Llama 3.3 70B $0.60/$1.20; DeepSeek-V3.1/V3.2 $3.00/$4.50; gemma-4-31B $0.38/$1.15; MiniMax-M3 $0.60/$2.40
DeepInfra competes almost entirely on price and does it credibly: DeepSeek-V4-Flash-0731 at $0.08/$0.18 per 1M, Llama-3.3-70B-Turbo at $0.10/$0.32, Llama-3.1-8B-Turbo at $0.02/$0.04, Qwen3.5-9B at $0.10/$0.15, Kimi-K2.6 at $0.75/$3.50, Kimi-K3 at $2.85/$14.25, Mistral-Nemo at $0.019/$0.03. Dedicated GPUs are the cheapest published rates among managed inference vendors — A100 $0.89/hr, H100 $2.20/hr, H200 $2.69/hr, B200 $3.69/hr, B300 $4.89/hr — roughly a third of Baseten's or Fireworks' list. It offers a base tier and a faster Turbo tier on the same models, so you choose the latency/price point per request.
| Output speed, gpt-oss-120b | 163 tok/s Turbo / 46 tok/s base (Artificial Analysis) |
|---|
| Models hosted | ~100+ across text, image, embedding, audio |
|---|
| Dedicated endpoints | Yes — dedicated GPUs from $0.89/hr (A100) |
|---|
| Llama/DeepSeek/Qwen-class price | DeepSeek V4 Flash $0.08/$0.18; Llama 3.3 70B $0.10/$0.32 per 1M |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | Small signup credit |
|---|
Watch out: The cheap tier is genuinely slow — Artificial Analysis measures 46 tok/s on base gpt-oss-120b versus 163 on Turbo — so the headline price and the usable price are different numbers. Support, status transparency and enterprise paperwork (SOC 2 scope, DPAs, regional residency) are thinner than at Fireworks, Baseten or Nebius, and there is no meaningful SLA on the shared tier. Wrong choice for a regulated buyer or a latency-critical interactive product.
DeepSeek V4 Flash $0.08/$0.18 per 1M; Llama 3.3 70B $0.10/$0.32; Llama 3.1 8B $0.02/$0.04; Kimi K3 $2.85/$14.25; dedicated H100 $2.20/hr, B200 $3.69/hr
Novita runs a broad, aggressively priced serverless catalog (200+ models covering LLMs, image, video and audio) alongside GPU instances, serverless endpoints and dedicated endpoints, and is one of the Hugging Face Inference Providers so you can call it straight from the Hub. Input-side rates as of September 2026 include GPT OSS 120B $0.05/1M, GPT OSS 20B $0.04/1M, Llama 3.1 8B $0.02/1M, Llama 3.3 70B $0.135/1M, Qwen3 Coder 30B $0.07/1M, Qwen3.8 Flash $0.15/1M, Qwen3.5 27B $0.30/1M, GLM 5.3 Flash $0.075/1M, DeepSeek V4 Flash $0.14/1M, DeepSeek V4 Pro $1.60/1M, Kimi K3 $3.00/1M. It sits in the same price band as DeepInfra with a wider multimodal catalog.
| Output speed, gpt-oss-120b | 150 tok/s (Artificial Analysis) |
|---|
| Models hosted | 200+ |
|---|
| Dedicated endpoints | Yes — serverless and dedicated endpoints, plus GPU instances |
|---|
| Llama/DeepSeek/Qwen-class price | Llama 3.3 70B $0.135/1M in; DeepSeek V4 Flash $0.14/1M in; Qwen3.8 Flash $0.15/1M in |
|---|
| OpenAI-compatible | Yes (also a Hugging Face Inference Provider) |
|---|
| Free tier | Trial credits |
|---|
Watch out: Throughput is low — 150 tok/s on gpt-oss-120b, roughly a third of Groq and a twelfth of Cerebras — so this is a batch and background-job platform, not an interactive one. Published pricing is largely input-side on the marketing page, so you must check output rates per model before modelling costs. Enterprise assurances, dedicated-endpoint rate cards and regional residency options are less developed than at Nebius or Baseten.
GPT OSS 120B $0.05/1M in; Llama 3.3 70B $0.135/1M in; DeepSeek V4 Flash $0.14/1M in; Qwen3 Coder 30B $0.07/1M in; Kimi K3 $3.00/1M in
Nebius rebranded AI Studio to Token Factory during 2026 (docs moved to docs.tokenfactory.nebius.com) and repositioned it as the enterprise inference tier of its GPU cloud. It serves 60+ open models — DeepSeek, Qwen, Llama, Mistral, GPT OSS, GLM and NVIDIA Nemotron — on an OpenAI-compatible API with shared and dedicated tiers, optional EU or US regional deployment, custom autoscaling and a 99.9% uptime SLA, and you can serve your own fine-tunes through the same dashboard. Blended prices range from $0.08/1M for Nemotron 3.5 Lightning (BF16) to $4.20/1M for Kimi K3, a 54x spread; Qwen3 32B Base is $0.12/1M blended, gpt-oss-120b Base runs 313 tok/s at $0.12/1M. Volume discounts start at 10% at 1M tokens/month and reach 50% at 1B/month, with a further 20-30% off via the batch API.
| Output speed, gpt-oss-120b | 307-313 tok/s (Artificial Analysis) |
|---|
| Models hosted | 60+ |
|---|
| Dedicated endpoints | Yes — 99.9% SLA, custom autoscaling, EU/US regions |
|---|
| Llama/DeepSeek/Qwen-class price | Qwen3 32B $0.12/1M blended; Qwen3 30B A3B $0.12/1M; Kimi K3 $4.20/1M |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | Signup credits; limited free models |
|---|
Watch out: The AI Studio to Token Factory rename means older docs, SDK references and blog links are stale — budget time for that. The catalog skews heavily toward NVIDIA Nemotron variants (the cheapest and fastest options are all Nemotron), so the models that make Nebius look good on price/speed charts may not be the ones you want. Throughput is mid-pack at 307 tok/s on gpt-oss-120b, and the flagship-model prices at the top of the range ($4.20/1M for Kimi K3) are above Together and DeepInfra.
$0.08/1M blended (Nemotron 3.5 Lightning BF16) to $4.20/1M (Kimi K3); Qwen3 32B $0.12/1M; gpt-oss-120b Base $0.12/1M; volume discounts 10% at 1M tok/mo rising to 50% at 1B/mo; batch API a further 20-30% off
Parasail orchestrates rented GPU capacity across 40+ data centres in 15+ countries and exposes it as serverless, dedicated or batch inference. Its distinguishing move is breadth by construction rather than curation: it advertises per-token access to essentially any open-weight model on Hugging Face (2M+), with about 40 curated endpoints (Qwen, Llama, Mistral, DeepSeek, gpt-oss) already warm. Serverless spans $0.01/1M for BGE-M3 input to $15.00/1M at the top end, with gpt-oss-20b at $0.04/1M input; batch is priced in bands by parameter count and precision, starting at $0.02-$0.04 per 1M for 0-4.1B models at FP4. Artificial Analysis measures 126 tok/s and $0.07/1M blended on gpt-oss-120b. Dedicated GPU reservations run B300 down to RTX 5090, billed per minute, rates on request. It raised a $32M Series A in April 2026.
| Output speed, gpt-oss-120b | 126 tok/s (Artificial Analysis) |
|---|
| Models hosted | ~40 curated endpoints; 2M+ HF checkpoints on demand |
|---|
| Dedicated endpoints | Yes — reserved GPUs billed per minute, rates on request |
|---|
| Llama/DeepSeek/Qwen-class price | gpt-oss-20b $0.04/1M in; range $0.01-$15.00/1M across models |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | Trial credits |
|---|
Watch out: Dedicated GPU pricing is quote-only, which makes it impossible to compare against DeepInfra or Runpod without a sales call. Throughput is slow at 126 tok/s on gpt-oss-120b, and because capacity is rented across many third-party data centres, latency, region and reliability characteristics vary more than at a vendor operating its own fleet. It is a much smaller company than Together or Fireworks, so weigh vendor risk before making it a single point of failure.
Serverless $0.01-$15.00 per 1M depending on model; gpt-oss-20b $0.04/1M in; gpt-oss-120b ~$0.07/1M blended; batch from $0.02-$0.04/1M for sub-4B models at FP4; dedicated GPU rates on request
Hyperbolic runs an open-access GPU marketplace aggregating supply from many operators, with a serverless inference API bolted on for about 25 open models — OpenAI-compatible chat, vision, image generation (FLUX, Stable Diffusion, LoRA) and audio (Melo TTS). Reported on-demand GPU rates in 2026 have moved substantially: H100 SXM quoted at $1.50/hr in June and $3.19/hr by August, H200 $2.40 to $3.99, B200 $3.50 to $5.99, with consumer cards from $0.16/hr (RTX 3070) and $0.30/hr (RTX 4090). Reserved clusters carry 3-12 month commitments with up to 40% discount. Serverless inference has been quoted at Llama 3.3 70B $0.40/1M, DeepSeek R1 $3.00/1M and Llama 3.1 405B $4.00/1M, with images from $0.01.
| Output speed, gpt-oss-120b | Not measured by Artificial Analysis |
|---|
| Models hosted | ~25 open models |
|---|
| Dedicated endpoints | Yes — reserved GPU clusters, 3-12 month commits, up to 40% off |
|---|
| Llama/DeepSeek/Qwen-class price | Llama 3.3 70B ~$0.40/1M; DeepSeek R1 ~$3.00/1M (catalog appears dated) |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | Signup credits |
|---|
Watch out: Confidence in these figures is low: Hyperbolic's own pricing page did not resolve during research and the numbers come from secondary aggregators, and the quoted GPU rates roughly doubled between June and August 2026 — assume anything here is stale. The serverless model list looks a generation behind (Llama 3.1 405B, DeepSeek R1 rather than V4), and Artificial Analysis lists a $0.21/1M blended price for gpt-oss-120b with no throughput measurement at all, which suggests thin capacity. Wrong choice for a production API where you need a stable catalog and predictable latency.
H100 SXM $1.50-$3.19/hr, H200 $2.40-$3.99/hr, B200 $3.50-$5.99/hr, RTX 4090 $0.30/hr (rates moved sharply during 2026); serverless Llama 3.3 70B ~$0.40/1M, DeepSeek R1 ~$3.00/1M; reserved clusters up to 40% off
Replicate's model is that every model is a versioned, containerised artifact (a Cog image) with a typed schema, so you can run community models, push your own, and fork someone else's without touching infrastructure. Billing is predominantly per GPU-second rather than per token: CPU $0.000100/s, T4 $0.000225/s ($0.81/hr), L40S $0.000975/s ($3.51/hr), A100 80GB $0.001400/s ($5.04/hr), H100 $0.001525/s ($5.49/hr), scaling to 8x H100 at $0.012200/s ($43.92/hr, committed-spend contracts only). Some language models are billed per token instead — DeepSeek-R1 is listed at $3.75/1M input and $0.01 per thousand output tokens. Private models normally bill setup, idle and active time; fast-booting fine-tunes bill active time only.
| Output speed, gpt-oss-120b | Not measured by Artificial Analysis |
|---|
| Models hosted | Thousands of community models (image, video, audio, LLM) |
|---|
| Dedicated endpoints | Yes — private/dedicated deployments, multi-GPU on committed spend |
|---|
| Llama/DeepSeek/Qwen-class price | Mostly per GPU-second; DeepSeek-R1 $3.75/1M input, $0.01/1k output |
|---|
| OpenAI-compatible | Partial — native Replicate API is primary; OpenAI-compatible endpoint for some LLMs |
|---|
| Free tier | Small trial allowance |
|---|
Watch out: Per-GPU-second billing is a poor fit for LLM chat traffic — you pay for cold starts and idle, and a sustained H100 at $5.49/hr is more than double DeepInfra's $2.20/hr for the same card. The LLM catalog and per-token coverage lag the dedicated text providers badly (DeepSeek-R1 rather than V4), throughput is not benchmarked by Artificial Analysis, and multi-GPU tiers require committed-spend contracts. Wrong choice as a primary LLM backend; right choice for diffusion and audio.
H100 $0.001525/s ($5.49/hr), A100 80GB $0.001400/s ($5.04/hr), L40S $0.000975/s ($3.51/hr), T4 $0.000225/s ($0.81/hr), 8x H100 $43.92/hr; DeepSeek-R1 $3.75/1M input
Modal is not a model catalog: it is a serverless container runtime where you decorate Python functions and it schedules them onto GPUs with sub-second cold starts, so you bring vLLM or SGLang and own the serving stack. Per-second rates as of September 2026: B200 $0.001736/s ($6.25/hr), H200 SXM $0.001261/s ($4.54/hr), H100 SXM5 $0.001097/s ($3.95/hr), A100 80GB $0.000694/s ($2.50/hr), L40S $0.000542/s ($1.95/hr), A10 $0.000306/s, T4 $0.000164/s, plus CPU at $0.0000131/core/s, memory at $0.00000222/GiB/s and storage at $0.09/GiB/month with 1 TiB free. Starter is $0/mo with $30 free credits, 3 seats and 10 concurrent GPUs; Team is $250/mo with $100 credits and 50 concurrent GPUs.
| Output speed, gpt-oss-120b | N/A — depends entirely on the stack you deploy |
|---|
| Models hosted | None — bring your own container |
|---|
| Dedicated endpoints | Yes by construction — every deployment is your own container |
|---|
| Llama/DeepSeek/Qwen-class price | No per-token pricing; H100 $0.001097/s (~$3.95/hr) |
|---|
| OpenAI-compatible | Only if you deploy an OpenAI-compatible server (vLLM/SGLang) |
|---|
| Free tier | $30/month credits, 3 seats, 10 concurrent GPUs |
|---|
Watch out: There is no per-token API and no model catalog — you are responsible for vLLM configuration, batching, quantisation, KV cache tuning and evaluating your own throughput, which is weeks of work you would not do on Together or Fireworks. GPU concurrency is capped at 10 on the free plan and 50 on the $250/mo Team plan, so scaling requires an Enterprise conversation. Wrong choice if you just want tokens out of a standard open model.
H100 SXM5 $0.001097/s (~$3.95/hr), H200 $0.001261/s (~$4.54/hr), B200 $0.001736/s (~$6.25/hr), A100 80GB ~$2.50/hr, L40S ~$1.95/hr; Starter $0/mo + $30 credits, Team $250/mo + $100 credits
Runpod sells two things — Serverless (autoscaling GPU workers with a queue, billed per second of active work) and Pods (rented VMs, split into cheaper Community Cloud and more reliable Secure Cloud). September 2026 serverless rates: B300 $9.98/hr, B200 $8.64/hr, H200 $5.93/hr, H100 $4.79/hr, A100 80GB $2.72/hr, L40S/6000 Ada $1.75/hr, RTX 4090 $1.10/hr, A5000/L4/3090 $0.69/hr. Pods are much cheaper: H100 SXM $2.69 community / $3.49 secure, H200 $3.59/$4.59, A100 PCIe $1.19/$1.59, L40S $0.79/$1.09, RTX 4090 $0.34/$0.74, A5000 $0.16/$0.27. There are also public per-token endpoints and per-image/per-request endpoints, but the catalog is small and secondary to the compute business.
| Output speed, gpt-oss-120b | N/A — depends on the stack you deploy |
|---|
| Models hosted | A small set of public endpoints; primarily bring-your-own-container |
|---|
| Dedicated endpoints | Yes — Pods (community and secure) plus reserved capacity |
|---|
| Llama/DeepSeek/Qwen-class price | No competitive per-token card; H100 pod $2.69-$3.49/hr |
|---|
| OpenAI-compatible | Only via the vLLM template or an OpenAI-compatible worker you deploy |
|---|
| Free tier | None ongoing |
|---|
Watch out: Community Cloud is aggregated third-party hardware with no meaningful availability guarantee — nodes can disappear and network/disk performance varies, so it is not a production substrate. Serverless cold starts on large models are painful because weights must load into VRAM, and you are still responsible for the whole serving stack. The public per-token endpoint catalog is thin and not a credible alternative to Together or DeepInfra.
Serverless H100 $4.79/hr, A100 $2.72/hr, RTX 4090 $1.10/hr; Pods H100 SXM $2.69 (community) / $3.49 (secure), A100 PCIe $1.19/$1.59, RTX 4090 $0.34/$0.74, A5000 $0.16/$0.27
fal is the generative-media counterpart to the text-focused platforms: it hosts open and licensed image and video models on an inference runtime tuned for diffusion and transformer-video workloads, and bills by output unit rather than tokens or GPU time. Representative September 2026 rates: Wan 2.5 at $0.05/second of video, Kling 2.5 Turbo Pro at $0.07/second, Veo 3 at $0.4/second, Ovi at $0.2/video; Seedream V4 at $0.03/image, Flux Kontext Pro at $0.04/image, Qwen image at $0.02/megapixel. Serverless GPU compute is also available from about $1.89/hr for H100 up to $4.49/hr for B300 if you need to run your own workflow. Output-unit billing means you can price a feature per user action without modelling GPU seconds.
| Output speed, gpt-oss-120b | N/A — no LLM serving |
|---|
| Models hosted | Hundreds of image/video/audio models |
|---|
| Dedicated endpoints | Yes — private serverless deployments and dedicated GPU compute |
|---|
| Llama/DeepSeek/Qwen-class price | N/A — media only; $0.02-$0.04/image, $0.05-$0.4/video-second |
|---|
| OpenAI-compatible | No — native fal queue/REST API |
|---|
| Free tier | Trial credits |
|---|
Watch out: It does not serve LLMs, so it cannot be your text-inference platform — it is strictly complementary to Together, Fireworks or DeepInfra. Per-output pricing is easy to reason about but expensive at volume compared with renting a GPU and running the same open weights yourself, and some of the headline models (Veo 3, Kling) are proprietary rather than open weights, so a portion of the catalog is not portable elsewhere. No tokens/sec metric applies.
Video $0.05/s (Wan 2.5), $0.07/s (Kling 2.5 Turbo Pro), $0.4/s (Veo 3), $0.2/video (Ovi); images $0.02-$0.04 each; GPU compute from $1.89/hr H100 to $4.49/hr B300
CoreWeave added a pay-per-token serverless inference product on top of its large GPU fleet, with tracing and evals built in, aimed at customers already buying reserved capacity who want an elastic overflow tier. Artificial Analysis measures it at 74 tok/s and $0.03/1M blended on gpt-oss-120b — the cheapest blended rate in that comparison. The underlying GPU business publishes instance-level hourly rates (GB200 NVL72 $42.00/hr, HGX B200 $68.80/hr, HGX H100 $49.24/hr for the full node, A100 $21.60/hr, L40S $18.00/hr in North America), but the per-token inference columns are largely 'contact sales'. Practically, this is an add-on for existing CoreWeave accounts rather than a self-serve developer API.
| Output speed, gpt-oss-120b | 74 tok/s (Artificial Analysis) |
|---|
| Models hosted | Small curated set; count not published |
|---|
| Dedicated endpoints | Yes — this is the core business (reserved GPU clusters) |
|---|
| Llama/DeepSeek/Qwen-class price | Not published — contact sales; gpt-oss-120b ~$0.03/1M blended |
|---|
| OpenAI-compatible | Yes |
|---|
| Free tier | None |
|---|
Watch out: Almost nothing is self-serve: the published pricing page routes per-token inference to 'contact sales', so you cannot benchmark cost without a commercial conversation, and confidence in the $0.03/1M figure is low because it comes from a third-party benchmark rather than a rate card. Throughput is the weakest in the comparison at 74 tok/s on gpt-oss-120b. The catalog is small relative to Together or Novita. Wrong choice for a startup that wants to swipe a card and start calling an endpoint today.
gpt-oss-120b ~$0.03/1M blended (Artificial Analysis); GPU nodes HGX H100 $49.24/hr, HGX B200 $68.80/hr, GB200 NVL72 $42.00/hr, A100 $21.60/hr, L40S $18.00/hr; most per-token rates are contact-sales
Lambda is a GPU cloud with among the cleanest published on-demand rates: H100 SXM $3.99-$4.29/hr, B200 SXM6 $6.69-$6.99/hr, GH200 $2.29/hr, A100 SXM/PCIe $1.99-$2.79/hr, plus 1-Click Clusters at $6.16/GPU/hr for 16x H100 down to $5.54 at 256 GPUs, and $9.86 down to $8.87/GPU/hr for HGX B200 configurations on two-week-to-one-year terms. The important 2026 change is that its OpenAI-compatible per-token Inference API is being wound down — Lambda's own inference page now directs customers to deploy on NVIDIA GPU instances instead. Treat Lambda as a compute vendor, not an inference API vendor.
| Output speed, gpt-oss-120b | N/A — inference API winding down |
|---|
| Models hosted | Effectively none going forward |
|---|
| Dedicated endpoints | GPU instances and 1-Click Clusters only |
|---|
| Llama/DeepSeek/Qwen-class price | No ongoing per-token card; H100 $3.99/hr |
|---|
| OpenAI-compatible | Was yes; API deprecated |
|---|
| Free tier | None |
|---|
Watch out: Do not build on the Lambda Inference API: it is explicitly winding down and Lambda has not published a shutdown date, model list or ongoing rate card, so any per-token integration you write here is throwaway work. As a pure GPU cloud there is no managed serving stack, no model catalog and no OpenAI-compatible endpoint unless you deploy one yourself. Reserved terms beyond one year require a sales conversation, and 1-Click Clusters carry a two-week minimum.
H100 SXM $3.99-$4.29/hr, B200 SXM6 $6.69-$6.99/hr, GH200 $2.29/hr, A100 $1.99-$2.79/hr; 1-Click Clusters 16x H100 $6.16/GPU/hr, 256x H100 $5.54/GPU/hr, 16x B200 $9.86/GPU/hr; Inference API winding down
· deprecated
OpenRouter is a routing and billing layer rather than an inference platform: one API key and one OpenAI-compatible endpoint reach 500+ models across 80+ providers, including most of the vendors in this chapter, with automatic failover when a provider degrades and per-provider routing preferences (cheapest, fastest, or a named list). Its practical value is discovery and de-risking — you can A/B the same open model across Together, DeepInfra, Novita and Parasail without four integrations, and you get normalised usage and cost analytics across all of them. Pricing is provider pass-through plus a platform fee; the marketing site does not publish the markup percentage, so confirm the current fee schedule and any BYOK terms in the docs before committing.
| Output speed, gpt-oss-120b | Varies by routed provider (46-1,849 tok/s across the pool) |
|---|
| Models hosted | 500+ models across 80+ providers (routed, not hosted) |
|---|
| Dedicated endpoints | No — routes to providers' endpoints; supports provider pinning and BYOK |
|---|
| Llama/DeepSeek/Qwen-class price | Provider pass-through plus undisclosed platform fee |
|---|
| OpenAI-compatible | Yes — this is the entire point |
|---|
| Free tier | Rotating free models with strict rate limits |
|---|
Watch out: You add a network hop and a second billing relationship to every request, which costs latency and makes p99 debugging harder — you are now diagnosing someone else's routing decision. The markup is not publicly documented, and routing to whichever provider is cheapest means quantisation, context window and throughput can silently change under you between requests unless you pin providers. It also gives you no leverage to negotiate committed-spend discounts, so it is the wrong permanent home for a workload you already understand.
Pass-through of each provider's per-token rate plus an undisclosed platform fee (not published on the marketing site); credits purchased in advance, e.g. $10 or $99