Every AI feature you ship eventually makes an API call to a model. The provider you choose decides your latency, your bill, and how much lock-in you accept. We evaluate eight platforms that sit between your code and the models: routers, inference platforms, model hubs, and raw GPU compute.

Quick picks at a glance

How we evaluate

We compare these platforms on the axes that actually change an implementation, not on a feature checklist:

• Pricing model. Per-token, per-second, per-GPU-hour, or a markup on provider rates — and how predictable the bill is at volume.
• Speed. Tokens per second and cold-start behavior for streaming or real-time features.
• Model breadth. How many models you can reach, and whether that includes non-LLM workloads (image, video, audio, 3D).
• Customization. Fine-tuning, LoRA hosting, and dedicated capacity for production.
• Control vs. ergonomics. Raw compute you fully own versus a one-line drop-in endpoint.

All ratings, pros, cons, and prices come from each tool’s directory profile. We do not run timed benchmarks; we compare what each platform is built to do and who it fits.

1. OpenRouter — best for routing across many providers

OpenRouter is a unified API gateway: one OpenAI-compatible key and one bill reach 300+ models from OpenAI, Anthropic, Google, Meta, and DeepSeek, with automatic failover when an upstream provider rate-limits or goes down.

• Pros: One key for 300+ models; pass-through pricing at provider rates plus a 5.5% credit fee; automatic failover keeps apps online during outages.
• Price: Free tier with 28+ $0/token models; pay-as-you-go at provider rate + 5.5% fee; BYOK at 5% past 1M free requests/mo.
• Skip it if: your workload calls a single provider at very high volume — the 5.5% markup then outweighs the routing convenience, and you have no SLA unless you negotiate a custom contract.
• Real example: a startup flips between Claude, GPT, and Llama by changing a model string, so a provider outage never takes its chat feature offline.

→ Full profile: OpenRouter

2. Together AI — best for cheap open-model inference with fine-tuning

Together AI runs 200+ open models (Llama, DeepSeek, Qwen, Mixtral, FLUX) through one OpenAI-compatible endpoint, with serverless inference and dedicated endpoints.

• Pros: Among the cheapest per-token open-model inference, often 5–10× below proprietary APIs; fine-tuning and deployment in one workflow; strong throughput (Llama-class models clear 200 tokens/sec).
• Price: $1–5 sign-up credits; serverless from $0.06/M tokens; dedicated endpoints from $0.85/hr; batch mode at 50% off.
• Skip it if: you need niche models outside its curated catalog, or heavy video/audio generation — it is text/code-first and has no true sustained free tier.

→ Full profile: Together AI

3. Fireworks AI — best for fast, fine-tunable production inference

Fireworks is an inference platform for teams shipping open-weight models to production, started by former Meta PyTorch and Google engineers. Its FireAttention engine delivers some of the highest tokens/sec in the industry.

• Pros: Among the fastest open-weight inference; fine-tuning and LoRA hosting in the same API; 50% batch discount for offline workloads; SOC 2, HIPAA, and GDPR with zero-retention options.
• Price: $1 starter credit; serverless $0.10–0.90/MTok; batch at 50% off; dedicated GPUs $2.90–9/hr.
• Skip it if: you need the absolute cheapest per-token rate — Fireworks is not the lowest, and only $1 of free credit limits how deep you can evaluate before paying.

→ Full profile: Fireworks AI

4. Groq — best for real-time, low-latency streaming

Groq runs models on its own LPU hardware at 500–1,000+ tokens/sec, a favorite for live chat, voice agents, and real-time RAG.

• Pros: Near-instant streaming at 1,000+ tokens/sec; pay-as-you-go per token with no monthly minimum; open-weight text models plus Whisper transcription and built-in web search.
• Price: $0 developer tier; production $0.05–$1.00/1M in; Whisper from $0.04/hr; cached input and Batch API cut cost roughly in half.
• Skip it if: you need private fine-tunes or reserved capacity — Groq rents speed, not control, and serious customization still points back to self-hosting.

→ Full profile: Groq

5. Replicate — best for 1,000+ community ML models

Replicate lets you run thousands of open-source ML models — Flux, Stable Diffusion, Wan, Whisper, Llama — through one REST API, with no GPU to manage. It scales to zero and bills per second.

• Pros: API access to 1,000+ community and official models; pay-per-second billing with scale-to-zero; Cog framework to package and deploy your own models as endpoints.
• Price: $5 sign-up credits; T4 $0.000225/sec; A100 80GB $0.0014/sec; ~$0.025–0.04/image for Flux-class models.
• Skip it if: you run latency-sensitive features at high volume — cold starts and hard-to-forecast per-second bills are the trade-offs. It is “Heroku for AI models,” not a guaranteed-latency layer.

→ Full profile: Replicate

6. Modal — best for arbitrary GPU compute and jobs

Modal is serverless compute for engineers who want GPUs, batch jobs, and web endpoints without managing a VM. You write a Python function, decorate it with resource hints, and Modal handles scheduling and teardown.

• Pros: Autoscale-to-zero means you rarely pay idle GPU time; one Python script deploys batch jobs, APIs, and cron together; broad GPU menu with per-second pricing.
• Price: Starter $0 ($30/mo credits); Team $250/mo + usage; H100 ~$0.0011/s, A100 80GB ~$0.00069/s.
• Skip it if: you want portable infrastructure — Modal’s decorator API is a lock-in, and the Team plan’s $250/month base can surprise smaller teams.

→ Full profile: Modal

7. Hugging Face — best for the open-model hub and endpoints

Hugging Face is the “GitHub for ML”: 1M+ models, 500k+ runnable Spaces, and Inference Providers routing 200+ models at no markup.

• Pros: Largest open repository with one-line loaders; Spaces to run and share 500k+ demos; Inference Providers pass through 200+ models at no markup plus dedicated endpoints.
• Price: Free tier; PRO $9/mo; Team $20/user/mo; Endpoints $0.50–$10/GPU-hr billed per minute while running.
• Skip it if: you need predictable total cost — between seats, endpoints, Spaces, and storage, a careless deployment surprises you at month end. Pause idle resources and watch the billing dashboard.

→ Full profile: Hugging Face

8. Cohere — best for RAG, embed, and rerank APIs

Cohere builds LLMs for retrieval: the Command family, Embed for semantic vectors, and Rerank for re-ordering retrieved passages.

• Pros: Best-in-class Rerank and Embed APIs that lift RAG quality; RAG-native models for retrieval, citation, and tool use; private and on-prem deployment for regulated industries.
• Price: Trial $0 (rate-limited); production pay-as-you-go (Command R $0.15/$0.60 per 1M in/out; Embed $0.10/M; Rerank ~$1–2/1K).
• Skip it if: you want a consumer chatbot — Cohere is wired into apps, not asked directly, and per-token reranking cost can dominate large pipelines.

→ Full profile: Cohere

How to choose

Pick by what your code actually needs:

• Multi-model app that must stay online → OpenRouter. One key, 300+ models, automatic failover.
• Open-weight LLM backend on a budget → Together AI. Cheapest per-token with fine-tuning built in.
• Live chat or voice agent → Groq. Lowest latency; Fireworks AI if you also need to fine-tune.
• Image, video, or audio models → Replicate. Broadest community model catalog.
• Custom training or spiky GPU jobs → Modal. Pay only for seconds run.
• Discover, benchmark, and host open models → Hugging Face. The default on-ramp.
• Retrieval-heavy RAG → Cohere. Rerank and Embed lead the field.

Related tools & guides

• Pinecone — vector database that pairs with any of these providers in a RAG stack.
• LangChain — orchestration layer that calls these APIs from your app.
• LlamaIndex — data framework for RAG on top of inference providers.
• v0 — Vercel’s AI app builder that ships against model APIs.
• Cursor — AI coding assistant for wiring provider SDKs into your codebase.
• Best AI Coding Assistants 2026 — the IDE tools that call these endpoints.
• AI Coding Assistants Benchmark 2026 — how we scored the assistants.
• Pinecone vs Weaviate vs Qdrant 2026 — choosing the vector store behind your RAG.