NVIDIA delivered a clean sweep in MLPerf Training v6.0, winning every benchmark on both overall performance and per-accelerator metrics, and standing as the only platform to submit results across all tests. The results were driven by the Blackwell Ultra-based GB300 NVL72 and the prior-generation GB200 NVL72, and demonstrate both leading absolute scale and rapid software-driven gains without hardware changes.

The GB300 NVL72 rack pairs 76 Blackwell Ultra GPUs with 38 Grace CPUs connected over NVLink, and NVIDIA scaled submissions to as many as 8,684 GPUs across production cloud deployments. On the largest and most demanding workloads the platform set records: DeepSeek-V3, a 671-billion-parameter mixture-of-experts model, trained to target on 8,684 GPUs in just 2.14 minutes. GPT-OSS 20B, also an MoE, trained in 7.88 minutes on 543 GPUs. Llama 3.3 405B reached target in 7.49 minutes on 8,684 GB200 NVL72 GPUs, and Llama 3.3 8B trained in 4.73 minutes on 1,085 GPUs.

Much of the improvement came from software. Six major optimizations delivered the gains. Full-iteration CUDA graphs for token-dropless MoEs eliminated CPU-GPU synchronization overhead. CuTe DSL kernel fusions contributed an 8% benefit on DeepSeek-V3 and a 99% speedup on GPT-OSS. MXFP8 attention blocks replaced 16-bit computation with 8-bit precision in the attention path. Router optimizations produced a 5x kernel speedup for a 5% end-to-end gain. A 1F1B all-to-all overlap achieved nearly 106% communication overlap for an 8% benefit, and pipeline stage balancing reduced imbalance to under 1% for an additional 4% of savings.

The cumulative effect on throughput is striking. DeepSeek-V3 training throughput on GB300 improved 1.3x over three months with no hardware changes, rising from 1,153 TFLOPS/GPU in the November 2025 (v25.11) software stack to 1,292 TFLOPS/GPU in February 2026 (v26.02), 1,376 TFLOPS/GPU in April 2026 (v26.04), and 1,747 TFLOPS/GPU in June 2026 (v26.06), equivalent to 6,718 tokens per second per GPU. This trajectory shows how the same silicon keeps getting faster through the software stack.

Generation-over-generation, the Blackwell Ultra-based GB300 pulled clearly ahead of the Blackwell-based GB200. GB300 delivered a 1.3x improvement on GPT-OSS 20B, 1.5x on Llama 3.3 405B, and 1.6x on DeepSeek-V3. The gains stem from GB300's increased memory capacity and larger power budget, which improve model locality and reduce cross-GPU communication overhead during large-scale distributed training.

The optimizations were delivered through NVIDIA's integrated software layers rather than one-off tuning. cuDNN provided MXFP8 attention through the Transformer Engine, CuTe DSL enabled the custom kernel fusions, and Megatron Core together with Megatron Bridge and cuBLAS supplied the training framework and dense linear algebra. Everything ships in the NVIDIA NeMo container, with v26.06 as the latest release, so the record-setting configurations are reproducible by customers running their own large-scale training jobs on Blackwell and Blackwell Ultra systems.