Microsoft announces Maia 300, its inference AI accelerator

Microsoft announced Azure Maia 300 on October 12, 2026, as the third generation of its AI accelerator program, with production deployments planned to begin in 2027. Designed for data-center inference, Maia 300 is a system built around the accelerator, its memory, networking, rack infrastructure and cooling—not a chip for running AI locally on a personal device. Microsoft’s announcement and technical architecture post describe the plan and specifications.

What Maia 300 is designed to do

Microsoft positions Maia 300 for inference: the computing work involved in generating responses from a trained AI model. Its stated target workloads include long-context reasoning, memory-intensive and decode-heavy tasks, latency-sensitive conversations, and multi-step agentic tasks.

The accelerator uses a multi-chiplet design, with compute chiplets built using TSMC N2P process technology, according to Microsoft. The company also describes enhanced MXFP4 and sub-3-bit storage formats, which use low-precision numbers to reduce memory and data-movement demands.

From accelerator to rack-scale system

Each Maia 300 accelerator has eight HBM4 memory stacks, 288 GB of capacity and peak memory bandwidth of 18 TB/s. Microsoft lists a 950 W thermal design power (TDP) for the accelerator. These are chip-level specifications.

A full rack brings together 96 accelerators across 24 trays, with four accelerators per tray. Microsoft gives the rack a 167 kW power envelope, 27.6 TB of HBM capacity and 1.7 PB/s of aggregate HBM bandwidth. The rack’s power envelope describes a different system scale from the accelerator’s 950 W TDP.

Microsoft describes a two-tier, Ethernet-based scale-up network that can connect as many as 12,288 accelerators in a superpod. Its cooling design uses liquid cooling for accelerators, voltage regulators and switch ASICs; air cooling serves memory, storage and ancillary components. Microsoft says liquid cooling removes more than 80% of the rack’s thermal load from the components it serves.

What Microsoft’s performance claims compare

Microsoft says Maia 300 has 2.5 times Maia 200’s memory bandwidth, 33% more memory capacity and about twice its HBM bandwidth per watt. These are comparisons with the previous Maia generation.

At rack scale, Microsoft claims that a full 96-accelerator Maia 300 configuration can deliver up to seven times Maia 200’s token throughput. Separately, the company expects Maia 300 to deliver 40% more tokens per megawatt than leading GPU-based systems. The first comparison is rack throughput against Maia 200; the second is tokens per unit of power against a broader GPU-system category.

Microsoft also describes up to 2.5 times the performance per watt versus Maia 200 at rack scale. Each figure reflects a different metric and comparison baseline, with the 2027 production-deployment start remaining the company’s announced plan.