Groq made the LPU, an inference chip that schedules every operation at compile time and keeps weights in SRAM. In December 2025 Nvidia took a non-exclusive licence to the technology in a deal widely reported at $20 billion, and founder Jonathan Ross and president Sunny Madra joined Nvidia. Groq stays independent under CEO Simon Edwards and keeps GroqCloud running.
Key Takeaways
- What it is — Inference specialist. Deterministic, compiler-scheduled inference chip with weights in on-chip SRAM.
- Best for — Low-latency, low-cost inference on popular open-weight models through a simple API.
- Watch out — After the Nvidia licensing deal the founders and much of the team moved to Nvidia; the roadmap now runs through Nvidia.
At a glance
| Company | Groq (technology licensed to Nvidia) |
|---|---|
| Current chips | LPU (language processing unit) in GroqCloud; Nvidia's Groq 3 LPX rack due in the second half of 2026 |
| Architecture | Deterministic, compiler-scheduled inference chip with weights in on-chip SRAM |
| Memory | On-chip SRAM per LPU; Nvidia's Groq 3 LPX rack carries 128 GB of SRAM across 256 LPUs |
| Scale-up | Many LPUs per model; Groq 3 LPX: 256 LPUs per liquid-cooled rack with 640 TB/s scale-up bandwidth |
| Software | OpenAI-compatible GroqCloud API for hosted open-weight models |
| Where to get it | GroqCloud API and dedicated deployments; Nvidia sells the LPX rack |
| Pricing | Per token. GPT-OSS-120B listed at $0.15 per million input and $0.75 per million output tokens (January 2026) |
What changed in 2026
- Dec 2025. Groq and Nvidia sign a non-exclusive inference technology licence; Jonathan Ross and Sunny Madra join Nvidia, GroqCloud continues.
- Mar 2026. Nvidia shows the Groq 3 LPU and LPX rack at GTC, for the second half of 2026, with LP35 in 2027 and LP40 in 2028.
Scores
Scored 1 to 5 on the same five axes as every vendor in the AI chip companies analysis.
| Performance | Scale-up | Software | Access | 2026 momentum | Total / 25 |
|---|---|---|---|---|---|
| 5 | 3 | 3 | 4 | 3 | 18 |
Verdict
GroqCloud is still one of the cheapest fast APIs for open-weight models. For a long-term hardware bet, treat Groq's architecture as part of Nvidia's roadmap now.
Alternatives
In the same lane (inference specialist):
- Cerebras: the fastest tokens per second on open-weight models, for agents and interactive products where latency is the product.
- SambaNova: enterprises and governments that want fast inference on their own premises with many models per rack.
In other lanes:
- Google TPU (hyperscaler custom silicon)
- AWS Trainium (hyperscaler custom silicon)
- AMD Instinct (merchant gpu)
- Tenstorrent (open hardware)
- Microsoft Maia (hyperscaler custom silicon)
Head-to-head comparisons
- Google TPU vs AWS Trainium vs Nvidia GPU: The three ways to train and serve frontier-scale models in 2026, scored on scale-up domain, software, access and price.
- Cerebras vs Groq vs SambaNova: The SRAM-first inference specialists compared on speed, token price, deployment options and company risk.
New to custom AI silicon? Our partner site CTAIO explains how TPUs, pods and GPU clusters differ in TPU vs GPU.
Frequently asked questions
What is Groq?
Groq made the LPU, an inference chip that schedules every operation at compile time and keeps weights in SRAM. In December 2025 Nvidia took a non-exclusive licence to the technology in a deal widely reported at $20 billion, and founder Jonathan Ross and president Sunny Madra joined Nvidia. Groq stays independent under CEO Simon Edwards and keeps GroqCloud running.
How do you get access to Groq?
GroqCloud API and dedicated deployments; Nvidia sells the LPX rack. Pricing: Per token. GPT-OSS-120B listed at $0.15 per million input and $0.75 per million output tokens (January 2026).
What software does Groq run?
OpenAI-compatible GroqCloud API for hosted open-weight models.
Is Groq a good alternative to Nvidia?
GroqCloud is still one of the cheapest fast APIs for open-weight models. For a long-term hardware bet, treat Groq's architecture as part of Nvidia's roadmap now. It fits best for: low-latency, low-cost inference on popular open-weight models through a simple API. The main caveat: after the Nvidia licensing deal the founders and much of the team moved to Nvidia; the roadmap now runs through Nvidia.
Sources
- Groq newsroom: Nvidia licensing agreement
- DCD: Nvidia roadmap after the LPU launch at GTC 2026
- The Next Platform: Cerebras and Groq token pricing
Facts verified 2026-10-31. Vendor performance figures are as stated by the vendor.
Ready to Find the Right AI Tools?
Browse our data-driven rankings to find the best AI tools for your team.