Overview
Groq is an inference platform built around its own Language Processing Unit (LPU), custom silicon designed specifically for transformer workloads rather than general-purpose GPUs. The result is extremely high tokens-per-second throughput — many models stream at 500 to 1,000+ tokens per second — which makes it a favorite for latency-sensitive products like live chat, voice agents, and real-time retrieval. You reach it through a standard OpenAI-compatible API at https://groq.com, so most existing SDKs and frameworks drop in with a single base-URL change. The catalog spans open-weight text models (Llama, Qwen, GPT-OSS, Kimi), Whisper speech recognition, text-to-speech, and a Compound system that wires in web search and code execution. In our evaluation Groq’s standout value is predictable, linear, per-token pricing with no seat tax: you pay only for what you generate, which keeps small projects nearly free while scaling cleanly. It is less suited to teams that need private fine-tunes or guaranteed reserved capacity, where a dedicated GPU cloud or self-hosted stack is the better fit. For the common case of ‘I need a fast model endpoint now,’ Groq removes almost all friction.
Key Features
- LPU-powered low latency — custom inference hardware delivers 500–1,000+ tokens/sec on many models, so streaming responses feel near-instant for chat and voice.
- OpenAI-compatible API — swap your base URL and keep existing SDKs, LangChain, or agent frameworks working unchanged.
- Open-weight model catalog — Llama, Qwen, GPT-OSS, and Kimi text models plus Whisper transcription and Orpheus text-to-speech.
- Built-in tools — compound systems add web search, website visits, and Python code execution without extra services.
- Transparent per-token billing — no monthly minimum or seat fee; cached input and Batch API cuts cost roughly in half.
Pricing
| Plan | Price | For |
|---|---|---|
| Free / Developer | $0 | Pay-as-you-go, generous free rate limits on small models |
| Production (per token) | $0.05–$1.00 / 1M in, $0.08–$3.00 / 1M out by model | Production apps billed purely on usage |
| Audio | Whisper Turbo $0.04/hr, V3 $0.111/hr | Speech transcription workloads |
| Enterprise | Custom | Reserved capacity, support, compliance |
Comparison
Compared to Exa, Groq is an inference engine rather than a search index — Exa returns retrieved documents, while Groq runs the model that reasons over them, and the two are frequently paired in the same RAG stack. Against Perplexity, Groq is the raw model layer underneath: Perplexity ships a finished cited-answer product, whereas Groq gives you the speed to build your own answering experience.