Speeds and feeds
Accelerator Catalog
A selectable comparison matrix for AI training and inference accelerators: memory capacity, HBM bandwidth, low-precision peaks, interconnects, power envelopes, and source-backed detail pages.
How to read this table
This page keeps chip-level and accelerator-level numbers separate from node, rack, and pod numbers where possible. Vendor peak FLOPS are useful for orientation, but sustained model throughput depends on kernels, parallelism, memory traffic, collective communication, and software maturity.
Dense and sparse tensor peaks are not interchangeable. FP4, FP6, FP8, BF16, and INT8 formats also differ across vendors, so this catalog is a starting point for analysis rather than a final benchmark ranking.
| Metric | Instinct MI300XAMD - CDNA 3 | Instinct MI325XAMD - CDNA 3 | Instinct MI355XAMD - CDNA 4 | Instinct MI455XAMD - CDNA 5 | H200 SXMNVIDIA - Hopper | B200 SXMNVIDIA - Blackwell | TrainiumAWS - NeuronCore-v2 | Trainium2AWS - NeuronCore-v3 | Cloud TPU v5pGoogle - TPU v5p TensorCore | Cloud TPU v6e / TrilliumGoogle - TPU v6e TensorCore |
|---|---|---|---|---|---|---|---|---|---|---|
| Memory | ||||||||||
| CapacityGB | 192 | 256 | 288 | 432 | 141 | 180 | 34.36 (32 GiB) | 103.079 (96 GiB) | 95 | 32 |
| Technology | HBM3 | HBM3E | HBM3E | HBM4 | HBM3E | HBM3E | HBM | HBM | HBM2e | HBM |
| HBM bandwidthTB/s | 5.3 | 6 | 8 | 23.3 | 4.8 | 8 | 0.82 | 2.9 | 2.765 | 1.6 |
| Compute | Dense | SparsePeak throughput | |||||||||
| FP4PFLOPS | Dense: NASparse: NA | Dense: NASparse: NA | Dense: 10.1Sparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: 9Sparse: 18 | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA |
| BLOCK FP4PFLOPS | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: 40.265Sparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA |
| INT4TOPS | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA |
| FP6TFLOPS | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA |
| BLOCK FP6PFLOPS | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: 20.133Sparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA |
| FP8PFLOPS | Dense: 2.61Sparse: 5.22 | Dense: 2.61Sparse: 5.22 | Dense: 5Sparse: 10.1 | Dense: 20.133Sparse: NA | Dense: 1.979Sparse: 3.958 | Dense: 4.5Sparse: 9 | Dense: NASparse: NA | Dense: 1.299Sparse: 2.563 | Dense: NASparse: NA | Dense: NASparse: NA |
| BLOCK FP8PFLOPS | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: 20.133Sparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA |
| INT8POPS | Dense: 2.6Sparse: NA | Dense: 2.6Sparse: NA | Dense: 5Sparse: NA | Dense: 5.033Sparse: 10.066 | Dense: 1.979Sparse: NA | Dense: NASparse: NA | Dense: 0.38Sparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: 1.836Sparse: NA |
| BF16PFLOPS | Dense: 1.3Sparse: NA | Dense: 1.3Sparse: NA | Dense: 2.5Sparse: NA | Dense: 5.033Sparse: 10.066 | Dense: 0.989Sparse: NA | Dense: 2.25Sparse: NA | Dense: 0.19Sparse: NA | Dense: 0.667Sparse: NA | Dense: 0.459Sparse: NA | Dense: 0.918Sparse: NA |
| FP32TFLOPS | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: 315Sparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA |
| INT32TOPS | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA |
| FP64TFLOPS | Dense: 81.7Sparse: NA | Dense: 81.7Sparse: NA | Dense: 78.6Sparse: NA | Dense: 5Sparse: NA | Dense: 34Sparse: NA | Dense: 40Sparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA | Dense: NASparse: NA |
| Platform | ||||||||||
| Vendor | AMD | AMD | AMD | AMD | NVIDIA | NVIDIA | AWS | AWS | ||
| Architecture | CDNA 3 | CDNA 3 | CDNA 4 | CDNA 5 | Hopper | Blackwell | NeuronCore-v2 | NeuronCore-v3 | TPU v5p TensorCore | TPU v6e TensorCore |
| Unit | OAM accelerator | OAM accelerator | OAM accelerator | GPU (per accelerator) | SXM GPU | SXM GPU | Cloud accelerator chip | Cloud accelerator chip | Cloud TPU chip | Cloud TPU chip |
| Form factor | OAM module | OAM module | OAM module | Enhanced Accelerator Module (EAM) | SXM | SXM | Trn1 instance chip | Trn2 instance chip | Cloud TPU slice chip | Cloud TPU slice chip |
| Launch | 2023-12-06 | 2024-10-10 | 2025-06-12 | 2026-07-23 | 2023-11-13 | 2024-03-18 | 2022-11-28 | 2024-12-03 | 2023-12-06 | 2024-12-11 |
| Interconnect | AMD Infinity Fabric - 8 links, 128 GB/s peak per link | AMD Infinity Fabric - 8 links, 128 GB/s peak per link | AMD Infinity Fabric - 7 links, 153 GB/s peak per link | UALoE (UALink over Ethernet) - 3.6 TB/s peak bidirectional scale-up per GPU; 600 GB/s peak bidirectional scale-out per GPU | NVIDIA NVLink - 900 GB/s per GPU | NVIDIA NVLink 5 / NVSwitch - 1.8 TB/s per GPU, derived from DGX B200 aggregate | NeuronLink-v2 - 384 GB/s inter-chip | NeuronLink-v3 - 1.28 TB/s per chip | ICI 3D torus - 1200 GB/s bidirectional per chip | ICI 2D torus - 800 GB/s bidirectional per chip |
| Power | 750 W peak TBP | 1000 W peak TBP | 1400 W TBP | Not published in cited AMD specifications | Up to 700 W configurable TDP | Platform dependent; DGX B200 is ~14.3 kW max for 8 GPUs | Not published per chip | Not published per chip | Not published per chip | Not published per chip |
| Software stack | ROCm | ROCm | ROCm | ROCm | CUDA, TensorRT-LLM, NVIDIA AI Enterprise | CUDA, TensorRT-LLM, NVIDIA AI Enterprise | AWS Neuron SDK | AWS Neuron SDK | JAX, XLA, TensorFlow, PyTorch/XLA | JAX, XLA, TensorFlow, PyTorch/XLA |
Per accelerator, vendor peaks. NA = not reported. Single reported values are unshaded.
Hardware Roofline
Selected devices plotted as peak throughput ceilings against arithmetic intensity on log-log axes.
Instinct MI300X1.3 PFLOPS peak, knee 245.28 FLOP/byte
Instinct MI325X1.3 PFLOPS peak, knee 216.67 FLOP/byte
Instinct MI355X2.5 PFLOPS peak, knee 312.5 FLOP/byte
Instinct MI455X5.03 PFLOPS peak, knee 216.01 FLOP/byte
H200 SXM989 TFLOPS peak, knee 206.04 FLOP/byte
B200 SXM2.25 PFLOPS peak, knee 281.25 FLOP/byte
Trainium190 TFLOPS peak, knee 231.71 FLOP/byte
Trainium2667 TFLOPS peak, knee 230 FLOP/byte
Cloud TPU v5p459 TFLOPS peak, knee 166 FLOP/byte
Cloud TPU v6e / Trillium918 TFLOPS peak, knee 573.75 FLOP/byte