Accelerator
Instinct MI300X
AMD CDNA 3 oam accelerator summary for training,
inference, and roofline-style performance analysis.
Back to Accelerator Catalog
Vendor AMD
Architecture CDNA 3
Unit OAM accelerator
Form factor OAM module
Launch 2023-12-06
Memory 192 GB HBM3
HBM bandwidth 5.3 TB/s
BF16 peak 1.3 PFLOPS
FP16 peak 1.3 PFLOPS
FP8 dense peak 2.61 PFLOPS
FP8 sparse peak 5.22 PFLOPS
FP4 dense peak n/a
FP4 sparse peak n/a
FP64 peak 81.7 TFLOPS
INT8 peak 2.6 POPS
Interconnect AMD Infinity Fabric - 8 links, 128 GB/s peak per link
Power 750 W peak TBP
Software stack ROCm Notes
- Large-memory CDNA 3 accelerator commonly discussed for memory-heavy LLM inference.
- FP8 and INT8 sparse peak values are 2x dense peak when structured sparsity is used.
Sources