NVIDIA claimed a clean sweep in MLPerf Inference v5.0, winning all available benchmarks on the data-center submission track with the Blackwell-based GB200 NVL72 rack-scale system and the individual B200 GPU. MLPerf Inference v5.0 introduced two new benchmarks relative to the prior round: a Llama 3.3 405B dense language model test at the server scenario and a Mixtral 8x22B mixture-of-experts test, both at 105.9% of FP32 accuracy. Together with the carried-forward Stable Diffusion XL, ResNet-50, BERT-Large, and GPT-J tests, the round provides the broadest coverage of production AI inference workloads that MLPerf has offered to date.

On Llama 3.3 405B, the GB200 NVL72 achieved 33,920 queries per second in the server scenario, a result that NVIDIA contrasts against Hopper-based H100 systems from the prior round: the GB200 delivers roughly 6x more throughput per rack on this workload. The gain comes from a combination of NVLink bandwidth eliminating the PCIe bottleneck that limits H100 SXM configurations, higher HBM3E memory capacity allowing the 405B model's weights to reside fully on GPU without CPU offload, and Blackwell's second-generation Transformer Engine with FP8 precision reducing memory traffic during decoding.

The Mixtral 8x22B result highlighted mixture-of-experts inference as a distinct optimization challenge. Routing tokens to the correct expert subset each forward pass creates irregular memory-access patterns that stress the interconnect and memory subsystem differently than dense models. NVIDIA reported a 3x throughput improvement over Hopper on this benchmark, attributing it to custom TensorRT-LLM MoE kernels that batch expert dispatch efficiently on Blackwell's SM architecture and the higher NVLink-C2C bandwidth available in the NVL72 configuration.

Beyond peak throughput, NVIDIA submitted results showing power efficiency gains. The GB200 NVL72 delivers approximately 2.1x more tokens per watt on Llama 3.3 405B than a comparable H100 SXM rack, a metric that has become increasingly important as data center operators cite power availability — not GPU count — as their primary capacity constraint. NVIDIA attributed the efficiency improvement to FP8 and NVFP4 precision support reducing arithmetic intensity, and to the Grace CPU's memory subsystem reducing DRAM refresh overhead during inference idle cycles.

For the edge submission track, NVIDIA entered the Jetson AGX Orin and a new Jetson Thor developer kit, with results on ResNet-50, BERT-Large, and the 3D U-Net medical imaging benchmark. Jetson Thor, the Blackwell-based successor to AGX Orin, delivered a 4x throughput improvement over its predecessor on BERT-Large, reflecting the Blackwell Tensor Core improvement for transformer inference at edge-device power envelopes of 64–106 watts.

MLPerf Inference v5.0 also received submissions from AMD with its MI300X GPU, Qualcomm with its Cloud AI 106 Ultra, and Intel with Gaudi 3. AMD's MI300X posted competitive numbers on ResNet-50 and BERT-Large, while NVIDIA led on the Llama 3.3 405B and Mixtral tests where NVLink's scale-out bandwidth is most consequential. Independent observers noted that MLPerf's standardized benchmark harness and accuracy constraints make the results more comparable across vendors than proprietary benchmarks, though the choice of workloads still favors architectures optimized for transformer inference over convolutional networks.