Overview
Together AI is an AI-native cloud founded in 2022 and headquartered in San Francisco, engineered for developers building with open-source and frontier models. It provides serverless inference, fine-tuning, and GPU clusters on a custom inference stack that the company says delivers 3.5x faster inference and 2.3x faster training than alternatives, with Llama 3.1 8B running at 200+ tokens/second and 70B models at 80+ tok/s. The catalog spans 100+ production-ready open-source models — Llama, DeepSeek, Qwen, Mistral, Gemma and more — accessible through OpenAI-compatible endpoints for drop-in migration. Serverless inference is billed per token with input and output priced transparently, batch mode cuts costs 50% for asynchronous workloads up to 30 billion tokens per model, and dedicated endpoints start around $0.85/hour with SLA-backed latency. A fine-tuning pipeline supports LoRA and full tuning that trains then deploys to serverless in a single workflow without weight export. It also offers self-service and reserved GPU clusters (H100/H200/B200) and a code interpreter, positioning it as a full runtime rather than just a model host.
Key Features
- Serverless inference for 100+ open-source LLMs (Llama, DeepSeek, Qwen, Mistral, Gemma) via OpenAI-compatible endpoints — drop-in replacement
- Custom inference engine delivering 3.5x faster inference than comparable stacks, with Llama 3.1 8B at 200+ tokens/sec
- Transparent per-token pricing — you know the exact cost before sending a request, no per-second GPU math
- Fine-tuning pipeline (LoRA and full) that trains then deploys to serverless in one workflow, no weight export
- Batch API at 50% off for asynchronous workloads up to 30B tokens per model — ideal for RAG prep and bulk eval
- Dedicated endpoints from $0.85/hr with SLA-backed latency, plus self-service and reserved GPU clusters (H100/H200/B200)
Pricing
| Model tier | Input / Output | Notes |
|---|---|---|
| Llama 3.1 8B | $0.18 / 1M tokens | Budget, low-latency, high-volume |
| Llama 3.3 70B | ~$0.88 / 1M tokens | Mid-tier general purpose, 128K ctx |
| DeepSeek-R1 | $3.00 / $7.00 / 1M tokens | Reasoning tier |
| Dedicated endpoint | from $0.85 / hr | Guaranteed capacity, no cold start |
Token prices shift often; confirm on together.ai/pricing before committing volume.
Comparison
vs. Replicate: Replicate’s strength is media generation (image/video/audio) across a huge community catalog; Together AI is built for fast, cost-efficient open-source LLM serving. If your app is chat or agents making many calls, Together is the sharper tool.
vs. Groq: Groq wins raw token speed on custom LPU hardware, but offers far fewer models. Together trades some latency for breadth — 100+ curated models and fine-tuning you can’t get on Groq.
vs. Hugging Face: HF is where models live and get trained; Together is the optimized runtime. Teams often train on HF and serve through Together’s inference stack for production throughput.