Nvidia still sells most of the world's AI compute, but 2026 is the year its customers stopped treating that as permanent. Anthropic contracted gigawatts of Google TPUs, OpenAI signed for 6 GW of AMD GPUs and 750 MW of Cerebras, and Nvidia paid to license Groq's inference technology. This guide scores the eight alternatives that matter, against an Nvidia baseline, on what a buyer can actually get.
Key Takeaways
- Four lanes — Hyperscaler custom silicon (Google TPU, AWS Trainium, Microsoft Maia), merchant GPU (AMD Instinct), inference specialists (Cerebras, Groq, SambaNova) and open hardware (Tenstorrent). Each lane answers a different question.
- The interconnect is the moat — An Ironwood TPU pod joins 9,216 chips on one network; an Nvidia NVLink rack joins 72 GPUs. Pod-scale interconnect, not peak FLOPs, is why frontier labs are buying TPUs.
- Inference split off — Cerebras, Groq and SambaNova keep weights in on-chip SRAM to sell speed. Nvidia's licence of Groq's technology in December 2025 shows how seriously the incumbent takes that lane.
- Software decides the switch — AMD's ROCm is the only stack close to CUDA. TPUs, Trainium and the specialists all need a compiler port or an API move. Price that in before the hourly rate.
The four lanes
The vendors are not interchangeable. Each lane solves a different problem, and the shortlist should start from the problem.
1. Hyperscaler custom silicon
Google, Amazon and Microsoft design their own chips to cut the cost of the models they serve and the capacity they rent. You reach them through one cloud. Google's TPU and AWS Trainium are rentable; Microsoft's Maia runs Azure's own services.
- Google TPU: large, steady training runs and high-volume serving of standard transformer models, by teams already on Google Cloud.
- AWS Trainium: teams already committed to AWS that want a cheaper path for large PyTorch training and inference.
- Microsoft Maia: nobody buys Maia directly. It matters because it lowers the cost of the Azure models you already pay for.
2. Merchant GPU
AMD is the only other company selling data center GPUs at rack scale. Because it is still a GPU, it is the lowest-friction way to reduce Nvidia dependence, and ROCm runs most PyTorch code with little change.
- AMD Instinct: memory-heavy inference and teams that want GPU-style flexibility without Nvidia allocation.
3. Inference specialists
Cerebras, Groq and SambaNova keep model weights in on-chip SRAM rather than HBM, which removes the memory-bandwidth bottleneck for token generation. They sell speed per user, mostly through OpenAI-compatible APIs.
4. Open hardware
Tenstorrent sells RISC-V-based accelerators with an open-source software stack at every size, and licenses its designs to other chipmakers. It trades peak bandwidth for openness and cost.
- Tenstorrent: teams that want to own and modify the full stack, and buyers too small to get Nvidia allocation.
Why the interconnect matters more than the chip
Chip for chip, Google's Ironwood TPU and Nvidia's B200 are in the same class: 192 GB of HBM each and a few petaflops of FP8. The difference is how many chips share memory-speed bandwidth. Nvidia's NVLink domain is a 72-GPU rack today and a 144-GPU rack in 2027. An Ironwood pod is 9,216 chips in one 3D torus, and Google says TPU 8t's Virgo network links about 134,000 chips in a single data center fabric. For frontier training, where tensor and expert parallelism span hundreds of chips, that is the binding constraint.
The inference specialists attack a different constraint. Generating tokens one at a time is limited by how fast weights move from memory to compute. Cerebras, Groq and SambaNova put weights in on-chip SRAM, which is why they lead speed charts and why Nvidia's own Groq 3 LPX rack carries 128 GB of SRAM across 256 LPUs.
The mechanism is explained in more depth in TPU vs GPU on our partner site CTAIO.
Comparison
Seven attributes across eight vendors plus the Nvidia baseline. Scroll the table sideways on small screens. All facts verified 2026-10-31; vendor performance figures are as stated by the vendors.
| Vendor | Lane | Current chips | Memory | Scale-up domain | Software | Where to get it | Pricing |
|---|---|---|---|---|---|---|---|
| Google TPU | Hyperscaler custom silicon | Ironwood (TPU7x) generally available; TPU 8t (training) and TPU 8i (inference) announced April 2026 | Ironwood: 192 GiB HBM at about 7.4 TB/s per chip. TPU 8i: 288 GB HBM plus 384 MB on-chip SRAM | Ironwood pod: 9,216 chips in a 3D torus over ICI and optical circuit switches, 1.77 PB shared HBM. TPU 8t superpod: 9,600 chips | JAX and PyTorch via the XLA compiler, vLLM for serving; TorchTPU (native PyTorch) in preview. TensorFlow not supported on Ironwood | Google Cloud (Compute Engine, GKE). Alphabet began selling TPU systems to third parties in 2026; Oracle resale from 2027 | On demand per chip-hour, US: v5e $1.20, Trillium $2.70, v5p $4.20, Ironwood $12.00 ($5.40 on a 3-year commitment) |
| AWS Trainium | Hyperscaler custom silicon | Trainium3 (3 nm) generally available since December 2025; Trainium2 in volume; Trainium4 announced | Trainium3: 144 GB HBM3e at 4.9 TB/s per chip | Trn3 UltraServer: up to 144 chips, 362 MXFP8 petaflops, 20.7 TB HBM3e; UltraClusters reach hundreds of thousands of chips | AWS Neuron SDK: PyTorch, vLLM, Hugging Face Transformers, TorchTitan and Ray; NKI for custom kernels | AWS only (EC2 Trn2 / Trn3 instances and UltraServers, SageMaker HyperPod, EKS) | Per instance-hour on EC2; AWS positions it on cost per token rather than publishing a per-chip rate |
| AMD Instinct | Merchant GPU | MI355X (288 GB HBM3E) shipping; MI450 series in the Helios rack ramping from the second half of 2026 | MI355X: 288 GB HBM3E. MI450: up to 432 GB HBM4 at 19.6 TB/s per GPU | Helios rack: 72 MI450 GPUs, about 31 TB HBM4 and 1.4 FP8 exaflops, built to Meta's Open Rack Wide spec | ROCm, with upstream PyTorch, vLLM and SGLang support; closer to CUDA than any ASIC, but not identical | Azure, Oracle Cloud, neoclouds and on premises through OEM servers | Per GPU-hour from clouds and neoclouds; typically priced below comparable Nvidia parts |
| Cerebras | Inference specialist | Wafer-Scale Engine 3 (WSE-3) in the CS-3 system | 44 GB of on-chip SRAM per CS-3; larger models are spread across several systems | Clusters of CS-3 systems; each CS-3 rated at 12.5 FP16 petaflops dense and about 23 kW | OpenAI-compatible inference API for hosted open-weight models; training stack for dedicated customers | Cerebras Cloud API, dedicated capacity, and on-premises systems | Per token. GPT-OSS-120B listed at $0.25 per million input and $0.69 per million output tokens (January 2026) |
| Groq | Inference specialist | LPU (language processing unit) in GroqCloud; Nvidia's Groq 3 LPX rack due in the second half of 2026 | On-chip SRAM per LPU; Nvidia's Groq 3 LPX rack carries 128 GB of SRAM across 256 LPUs | Many LPUs per model; Groq 3 LPX: 256 LPUs per liquid-cooled rack with 640 TB/s scale-up bandwidth | OpenAI-compatible GroqCloud API for hosted open-weight models | GroqCloud API and dedicated deployments; Nvidia sells the LPX rack | Per token. GPT-OSS-120B listed at $0.15 per million input and $0.75 per million output tokens (January 2026) |
| SambaNova | Inference specialist | SN40L reconfigurable dataflow unit (RDU) | Three tiers per chip (on-chip SRAM, HBM and DDR), so one rack can hold many large models and switch between them | Rack-scale systems, sold as cloud capacity or installed on premises | SambaNova Cloud API for open-weight models and an enterprise stack for private deployments | SambaNova Cloud, plus dedicated and on-premises deployments | Per token on SambaNova Cloud; enterprise and on-premises deals quoted |
| Tenstorrent | Open hardware | Wormhole and Blackhole accelerators, sold in Galaxy servers | GDDR-based accelerators, which keeps cost down at the expense of HBM-class bandwidth | Galaxy servers scale from single cards and workstations to racks and superclusters over Ethernet | Open-source stack (TT-Metalium, TT-Forge) running open-weight models | Direct hardware sales and clouds such as Cirrascale | Published list prices for cards, workstations and Galaxy servers; cloud pricing through partners |
| Microsoft Maia | Hyperscaler custom silicon | Maia 200 (TSMC 3 nm), announced January 2026 | 216 GB HBM3e plus 272 MB on-chip SRAM per chip | Four accelerators per tray, 2.8 Tb/s bidirectional scale-up bandwidth per chip over Ethernet | Internal Azure stack serving Microsoft and OpenAI models | Not rented directly; you reach it through Azure AI services and Copilot that run on it | No per-chip price; Microsoft claims 30% better performance per dollar than its prior hardware |
| Nvidia GPU (baseline) | Merchant GPU (incumbent) | Blackwell (B200, GB200 NVL72, GB300) shipping; Vera Rubin NVL72 in the second half of 2026; Rubin Ultra Kyber NVL144 in 2027 | B200: 192 GB HBM3e | 72 GPUs per NVLink rack today; 144 GPUs with Kyber in 2027 | CUDA and every major framework and library | Every hyperscaler, neocloud and on premises | Per GPU-hour; wide spread between hyperscalers and neoclouds |
Scores
Five axes, each scored 1 to 5: chip-level performance for its target workload, scale-up domain, software maturity relative to CUDA, how easily a typical company can get access, and 2026 commercial momentum.
| Vendor | Performance | Scale-up | Software | Access | 2026 momentum | Total / 25 |
|---|---|---|---|---|---|---|
| Google TPU | 5 | 5 | 3 | 3 | 5 | 21 |
| AMD Instinct | 4 | 4 | 4 | 4 | 4 | 20 |
| Cerebras | 5 | 3 | 3 | 4 | 5 | 20 |
| AWS Trainium | 4 | 4 | 3 | 3 | 4 | 18 |
| Groq | 5 | 3 | 3 | 4 | 3 | 18 |
| Tenstorrent | 3 | 3 | 3 | 4 | 4 | 17 |
| SambaNova | 4 | 3 | 3 | 3 | 2 | 15 |
| Microsoft Maia | 4 | 3 | 2 | 1 | 3 | 13 |
Verdict by vendor
- Google TPU. The strongest second supply line for any team running large transformer workloads in JAX or PyTorch. Choose it when the job is big enough for pod-scale interconnect to matter and you can live inside Google Cloud.
- AWS Trainium. The default custom chip for AWS-native teams. It wins on price for large, standard PyTorch jobs, and loses wherever you need multi-cloud portability.
- AMD Instinct. The lowest-friction way to diversify away from Nvidia, because it is still a GPU. Validate Helios delivery dates in writing before you plan around them.
- Cerebras. The pick when output speed matters more than model choice. Use it as an inference tier next to your GPU or TPU estate, not as a replacement for it.
- Groq. GroqCloud is still one of the cheapest fast APIs for open-weight models. For a long-term hardware bet, treat Groq's architecture as part of Nvidia's roadmap now.
- SambaNova. A credible on-premises inference option when data residency rules out public APIs. Diligence the balance sheet as carefully as the benchmarks.
- Tenstorrent. The choice for builders who value an open stack and ownership over peak performance. Pilot it on a workload you can benchmark end to end.
- Microsoft Maia. Not a procurement option, but a signal: Azure inference pricing now has a custom-silicon floor under it, as Google's and AWS's do.
Head-to-head comparisons
How to choose
- Name the workload. Frontier-scale training, high-volume batch inference and latency-bound interactive inference have different winners.
- Start from your cloud. If you are on Google Cloud, the TPU is the obvious second supply line; on AWS, Trainium; on Azure, AMD and the Maia-backed services.
- Inventory your CUDA dependency. List the custom kernels and CUDA-only libraries. That list is the switching cost.
- Benchmark cost per token or per step. Run a short paid benchmark on your own model. Per-chip prices are not comparable across architectures.
- Diligence the vendor. For the startups, check customer concentration, funding and who owns the roadmap. For AMD, get delivery dates in writing.
Related reads
Frequently asked questions
Who are Nvidia's biggest competitors in AI chips?
In 2026 the strongest competitors are Google's TPU, which Anthropic, Meta and reportedly OpenAI are buying at scale; AMD's Instinct GPUs, which OpenAI has agreed to deploy at 6 GW; and AWS Trainium, which powers Anthropic's Project Rainier. Cerebras and Groq compete in fast inference, and Groq's technology is now licensed to Nvidia itself.
What is an AI accelerator?
An AI accelerator is a chip designed to run neural network workloads faster or cheaper than a general-purpose CPU. GPUs are the most common kind. Others are application-specific chips such as Google's TPU, AWS Trainium and Microsoft Maia, and specialised inference chips such as Cerebras' wafer-scale engine and Groq's LPU.
Is a TPU better than an Nvidia GPU?
For large training and serving jobs on standard transformer models in JAX or PyTorch, a TPU pod is often cheaper per unit of work because thousands of chips share one high-bandwidth network. For custom CUDA code, mixed workloads, or deployment outside Google Cloud, the Nvidia GPU remains the safer default. Our partner site CTAIO explains the trade-off in its TPU vs GPU guide.
Which AI chip is fastest for inference?
On public benchmarks of open-weight models, Cerebras and Groq lead on output tokens per second because they keep weights in on-chip SRAM. Artificial Analysis measured about 2,700 tokens per second for GPT-OSS-120B on Cerebras. Speed per user is not the same as cost per token at high batch sizes, where GPUs and TPUs are often cheaper.
Can I buy Google TPUs?
Until 2026 TPUs were only available to rent in Google Cloud. In 2026 Alphabet began selling TPU systems to third parties, and Oracle announced in October 2026 that it will resell TPU capacity, with customer availability expected in 2027.
What happened to Groq?
In December 2025 Groq signed a non-exclusive licence of its inference technology to Nvidia, in a deal widely reported at about $20 billion. Founder Jonathan Ross and president Sunny Madra joined Nvidia. Groq remains an independent company under CEO Simon Edwards and continues to run GroqCloud.
Sources are listed on each vendor page. Search volumes: DataForSEO, US, October 2026.
Ready to Find the Right AI Tools?
Browse our data-driven rankings to find the best AI tools for your team.