Microsoft details Azure Maia 300 chip architecture and claimed performance gains

The short version

• Microsoft unveiled Maia 300, its third-generation custom AI accelerator system for cloud-scale agentic inference, according to Hacker News.[Hacker News]
• Microsoft says the chip raises compute FLOPs 36%, HBM bandwidth 2.5 times, and scale-up bandwidth 14% over prior generations.[Hacker News]
• A 96-accelerator, 167 kW rack doubles Maia 200 accelerator density, and a two-tier network scales superpods to 12,288 accelerators.[Hacker News]
• Claimed gains of up to 7 times rack-scale token throughput and over 2.5 times performance per watt come from Microsoft's own comparisons; the reporting cites no independent benchmarks and does not state availability timing.[Hacker News]

Key facts

• Microsoft detailed Maia 300 as its third-generation accelerator system, designed for cloud-scale agentic inference.[Hacker News]
• Maia 300 increases compute FLOPs by 36%, HBM bandwidth by 2.5 times, and scale-up bandwidth by 14% compared with previous generations.[Hacker News]
• Each accelerator combines software-managed on-die SRAM with eight HBM4 stacks providing 288 GB of capacity and 18 TB/s of bandwidth.[Hacker News]
• A 167 kW rack holds 96 accelerators in 24 four-accelerator trays, doubling the accelerator density of Maia 200.[Hacker News]
• A two-tier network builds a superpod of up to 12,288 accelerators.[Hacker News]
• Relative to Maia 200, Maia 300 delivers up to 7 times rack-scale token throughput and more than 2.5 times performance per watt, according to Microsoft's figures as reported.[Hacker News]

Sources

Outlet counts describe coverage, not independent confirmation. Reports may share a wire service or original source.

• Deep dive into Azure Maia 300 architectureHacker News