The round at a glance

MLPerf® Training v6.1 expands the benchmark suite and features even larger systems from our members. Two benchmarks join the suite: Qwen3.5-397B GRPO for agentic reinforcement learning, and DLRMv4-HSTU, a generative recommender. Google’s Ironwood and NVIDIA’s Rubin make their debut as Available systems. The largest jobs reached 10,240 GPUs (NVIDIA on GB300 and Azure on GB200, both training DeepSeek-V3) [1, 2], multi-node systems rose to 76% of the total, and the hardware mix widened to 12 accelerator products and four scale-out fabric technologies. Submitters’ objectives also broadened, from characterizing new silicon to validating multi-tenant operation on an academic cluster.

The v6.1 round includes a total of 250 submissions from 25 submitters compared with 229 submissions from 24 submitters in v6.0. Results here.

This analysis uses verified MLPerf Training v6.1 Closed-division results published on November 16th at 8am PT throughout.

AI Training Performance Races Ahead of Moore’s Law

Moore’s Law has been synonymous with rapid technological improvements for decades and has driven benefits all across society. However, AI is truly a full-stack endeavor and the pace of progress is even faster. The chart below shows the evolution of an MLPerf Training performance index against a hypothetical Moore’s Law where performance doubles every two years.

Cumulative improvement vs Moore's law, v0.6 to v6.1

MLPerf Training v6.1

Chained geometric mean of gains on benchmarks common to consecutive rounds; Closed, Available, RCP-passing. Logarithmic y-axis. v0.5 omitted for rule changes.

Show exact chart data

Source: MLCommons historical results database and final MLPerf Training v6.1 results. Power/reference rows excluded. No external data requests.

The performance index in the chart is the geometric mean of the gains across the benchmark suite for the fastest Closed MLPerf Training result in each benchmark compared against the fastest result in the prior round. This index is not an official result and is derived from MLPerf results. For example, in v6.1 there are four benchmarks that carry over from the prior v6.0 round without changes to the benchmark definition and the geometric mean of the performance gains is 1.49.

Zooming in on this comparison, the chart below shows the best performance for the four comparable benchmarks from v6.0 and v6.1. Incidentally, these systems are all the same size, with 128 nodes and 512 accelerators, except for Llama 3.1 8B, where the v6.0 result had 256 nodes and 1,024 accelerators. 

This comparison highlights how the gains are often somewhat unevenly distributed. In about half a year the performance for FLUX.1 improved quite rapidly, while the gains for the smaller Llama 3.1 8B were much more muted.

Benchmark Updates

This round introduced two new benchmarks. As with any new benchmark, their first-round result counts partly reflect novelty:

  • Qwen3.5-397B GRPO: An agentic post-training workload that uses reinforcement learning to teach a 397-billion-parameter language model to repair real software projects. Seven submitters ran it, with 29 submissions.
  • DLRMv4 (HSTU): A generative recommendation model, replacing the older DLRM-DCNv2. Eight submitters made 20 submissions.

Two benchmarks were retired:

  • Llama 3.1 405B, a large dense pretraining model, has been retired after 3 rounds. While we prefer to keep most benchmarks for 2 years (4 rounds), we believe that large dense models are no longer representative of large-scale pretraining, which has shifted toward mixture-of-experts (MoE) models that activate only part of the model for each token. DeepSeek-V3 and Qwen3.5-397B now cover large-scale MoE training in the suite.
  • DLRM-DCNv2 has been retired after 7 rounds (3 years), replaced by DLRMv4 (HSTU).

Benchmark popularity, v5.1 to v6.1

MLPerf Training v6.1

Results: all divisions/availabilities. Organizations: Closed, Available, RCP pass. New benchmarks have no earlier-round comparison.

Show exact chart data

Source: MLCommons historical results database and final MLPerf Training v6.1 results. Power/reference rows excluded. No external data requests.

Who submitted, and why

Submitters also differ in what they use the benchmark to demonstrate:

  • Chip vendors (AMD, Google, NVIDIA) used the round to debut and characterize new silicon: first results for NVIDIA Vera Rubin and Google Ironwood, and broader workload coverage for AMD Instinct MI355X and MI350X.
  • Server OEMs and ODMs (ASUSTeK, Cisco, Dell, Fujitsu, GigaComputing, HPE, Inventec, MiTAC, Quanta Cloud Technology, Supermicro) demonstrate performance across different configuration choices, including accelerators, host CPUs, cooling systems, network fabrics and rack sizes. Together they account for 38 systems and 33% of all submissions.
  • Cloud providers (Azure, CoreWeave, Crusoe, Firmus, Google, Lambda, Nebius, Oracle) show that production infrastructure, often virtualized, can post competitive Closed results. 28% of all submissions run on systems listed as available in the cloud.
  • Academia: The University of Florida ran its benchmarks on half of the B200 SuperPOD in its HiPerGator cluster while the other half kept serving researchers. This illustrates that MLPerf results can be produced on shared academic infrastructure and demonstrates the performance achieved by real customers.
  • Software Optimizations: Firmus and KRAI are some of the organizations that demonstrated notable software improvements over baselines (including the use of AI-assisted optimizations).
  • Energy measurement. Korea’s TTA submitted the round’s only energy measurement.

“We submitted with power because energy is becoming as meaningful a dimension of AI infrastructure as time.” — TTA

Multi-node systems are the norm

Scale trends, v4.0 to v6.1

MLPerf Training v6.1

Counted per system; accelerator/node counts populated from v4.0 onward.

Show exact chart data

Source: MLCommons historical results database and final MLPerf Training v6.1 results. Power/reference rows excluded. No external data requests.

Multi-node systems and systems with 1,000+ accelerators by round, v4.0 to v6.1 (counted per system).

Multi-node systems are now the norm in MLPerf Training. 76% of all systems (117 of 153) used two or more nodes, up from 61% (72 of 118) in v6.0, and 19 of 25 submitters entered at least one multi-node system, up from 16 of 24. The typical system also grew: the median accelerator count per Closed result was 8 in every round from v4.0 through v6.0, and 32 in v6.1. Results using 64 or more accelerators rose from 73 in v6.0 to 111.

  • The largest Closed systems grew from 8,192 GPUs in v6.0 to 10,240 GPUs in v6.1: NVIDIA’s GB300 NVL72 and Azure’s GB200 NVL72 systems, both training DeepSeek-V3 [1, 2], while Google ran DeepSeek-V3 on 8,192 TPU7x (Ironwood) chips in a single ICI slice [10].
  • Cisco ran DeepSeek-V3 on 64 B300 GPUs (eight 8-GPU servers), the smallest system to run DeepSeek-V3 in v6.0 or v6.1 [11], giving a reference point for training a 671B-parameter MoE model on conventional servers rather than rack-scale systems.

System details

Accelerators

Twelve accelerator products appear in v6.1, from three vendors:

  • NVIDIA: H100, H200, GH200, B200, B300, GB200, GB300 and VR200, on 122 systems from 21 submitters. The largest submissions are Azure’s 10,240-GPU GB200 system and NVIDIA’s 10,240-GPU GB300 system [1, 2].
  • AMD: MI300X, MI350X and MI355X, on 27 systems from 9 submitters. The largest AMD system in v6.1 is Crusoe’s 128-GPU MI355X system on Llama 3.1 8B [12], running over standard Ethernet.
  • Google: TPU7x, on 4 systems of up to 8,192 chips.

Node design is shifting toward rack-scale systems. Half of v6.1 results (125 of 250) ran on 4-accelerator nodes, 115 of them on the compute trays of GB200, GB300 and Vera Rubin NVL72 racks; the other half ran on 8-accelerator servers, either as single nodes or as building blocks for multi-node systems of up to 256 accelerators (60 of the 125). In v6.0, 104 of 229 results used 4-accelerator nodes, and in v4.0, 35 of 181.

Systems by accelerator family, v6.0 vs v6.1

MLPerf Training v6.1

Accelerator names normalized to family. Counted per system; all divisions and availabilities.

Show exact chart data

Source: MLCommons historical results database and final MLPerf Training v6.1 results. Power/reference rows excluded. No external data requests.

Host CPUs

Host CPUs come from three families in v6.1: Arm-based NVIDIA Grace (64 systems) and, new this round, NVIDIA Vera (23 systems, paired with Vera Rubin); Intel Xeon (35); and AMD EPYC (31). In v6.0 the split was Grace 58, Xeon 34 and EPYC 26. Systems include both air- and liquid-cooled designs, and several OEMs submitted the same accelerator in multiple server configurations to show that results hold across CPU platforms and cooling options.

CPUs are also becoming part of training itself. In agentic RL such as Qwen3.5 GRPO, CPUs run the environments and tests that turn model responses into rewards. Submissions ran from about 500 to roughly 3,150 concurrent sandboxes per workload (with a 4,096 cap on a shared CPU pool), either on the accelerator nodes’ own host CPUs or, in Google’s case, on separate CPU nodes. We expect host CPU capacity to play a greater role as agentic workloads grow.

Network fabrics

v6.1 submissions use a wide range of interconnects:

  • Scale-up: NVIDIA NVLink, AMD Infinity Fabric, and Google ICI are extremely high-performance networks that are used to build tightly coupled nodes.
  • Scale-out: NVIDIA InfiniBand, Ethernet with RoCEv2 (including NVIDIA Spectrum-X and Cisco’s four-plane design), HPE Slingshot and Google DCN, with NICs from NVIDIA, AMD Pensando, Broadcom, HPE and Google. The scale-out network is used to connect multiple nodes together in a distributed system.
  • Scale-up domains: on conventional servers, the scale-up domain is a single node of 4 or 8 accelerators, so multi-node jobs use the scale-out network. On GB200, GB300 and Vera Rubin NVL72 systems and Google TPU7x slices, it spans multiple nodes (72 GPUs across 18 trays, or up to 8,192 TPU chips), so multi-node jobs can stay on the scale-up interconnect.

These choices matter more as jobs grow. The largest v6.1 runs span 160 NVL72 racks or an entire 8,192-chip TPU7x slice, so how a job divides its traffic between the scale-up and scale-out networks increasingly determines how fast it trains.

The complexity of Large Scale Training

More nodes do not automatically mean faster training. MLPerf Training measures time to train (TTT) with a fixed dataset and quality target, so a larger system has to finish the same job sooner. This is strong scaling, unlike HPL (the benchmark behind the TOP500 list), where the problem grows with the system. 

Using thousands of accelerators means partitioning both the computation and the model state across them. The two network tiers described above determine where each partition can go. Each partitioning scheme has a different communication pattern that stresses the system in distinct ways, and large jobs invariably combine several of them:

What limits scaling?

In theory, each accelerator can work independently on different sequences, followed by an all-reduce to synchronize the gradients. But that synchronization does not shrink as accelerators are added, while each accelerator’s share of the work does, so it takes up a growing fraction of every step: 100 accelerators will not train 100x faster. In practice, additional constraints add further overhead:

  • Limited global batch size. Each benchmark caps the number of sequences per training step so that training converges in a comparable number of samples, and that caps data parallelism. Llama 3.1 8B, Llama 2 70B LoRA, and GPT-OSS-20B process only about 64 sequences per step, so on hundreds of accelerators submitters must split each sequence across accelerators with tensor and context parallelism. That adds communication inside every layer while leaving less work for each accelerator.
  • Limited memory capacity. Even when the batch is large, as with DeepSeek-V3’s roughly 16,000 sequences per step, each data-parallel replica would need to hold the full model state: more than 10 TB for 671 billion parameters with weights, gradients and FP32 optimizer state, against 186–288 GB of HBM per accelerator. Even a whole node, with HBM and host memory combined, falls short. Submitters therefore split the model with pipeline and expert parallelism, adding point-to-point and all-to-all traffic, and recompute activations to save memory at the cost of extra compute.
  • Leaving the scale-up domain. Tensor, context and expert parallelism exchange data inside every layer, so submitters keep them within NVLink or ICI. Once a job outgrows a single scale-up domain, pipeline activations and gradient synchronization cross the scale-out network, which has roughly an order of magnitude less bandwidth per accelerator. Collective libraries soften this by spreading data-parallel traffic across all the NICs in a scale-up domain, so the effective bandwidth is set by the slower of the scale-up network and the combined NIC bandwidth of the replicas that share a domain.
  • Workload phases. Reinforcement learning adds a rollout phase that runs on inference engines and scales differently from the training step, and it must move updated weights to the rollout workers after every step.

The chart below shows how these constraints play out. DeepSeek-V3’s large batch leaves room for data parallelism on top of pipeline and expert parallelism, so it keeps getting faster all the way to 10,240 GPUs: on GB300, going from 256 to 1,024 GPUs (4x) cut time-to-train 3.9x [13, 14], and going on to 10,240 GPUs (10x more) cut it another 5.8x [1]. Llama 3.1 8B runs out of batch at a few dozen accelerators, and its curves flatten: on Vera Rubin, going from 72 to 512 GPUs (about 7x) cut time-to-train only 1.9x [3, 15]. These ratios are MLCommons-derived from NVIDIA’s Closed results on the same accelerator at each scale, not official MLPerf results.

  • Data parallelism (DP) replicates the model and splits the global batch across replicas, synchronizing gradients each step with an all-reduce or, when weights and optimizer state are sharded (ZeRO/FSDP), with reduce-scatter and all-gather. It is the simplest to scale, but the global batch size caps the DP degree.
  • Tensor parallelism (TP) splits individual weight matrices across devices, so every layer needs collective operations in both the forward and backward pass. It needs very high bandwidth and low latency, and in practice stays inside the scale-up domain.
  • Pipeline parallelism (PP) assigns contiguous groups of layers to different devices and passes activations point-to-point between stages. Its cost is the pipeline “bubble”, which is reduced by running many microbatches and interleaving several virtual stages per device.
  • Expert parallelism (EP) distributes the experts of a mixture-of-experts (MoE) model across devices. Every MoE layer requires two all-to-all exchanges (dispatch tokens to experts, then combine the results), which are bandwidth-heavy and irregular because routing is data-dependent.
  • Sequence and context parallelism (SP/CP) split the sequence dimension. SP shards activations in the parts of a layer that TP does not cover; CP splits attention itself across devices and exchanges key/value blocks between them. Both add parallelism independent of the batch size.

What strategies have been used?

A mixture-of-experts model can use different parallelism in its attention and MoE layers. In DeepSeek-V3, the attention and other dense components hold about 17 billion of the 671 billion total parameters (under 3%). Every token passes through them, though, so they make up nearly half of the 37 billion parameters active per token. The routed experts hold the other 97% of total parameters, but each token uses only 8 of the 256 experts in each layer. As a result, dense layers are usually replicated or lightly sharded, with data parallelism spreading the batch across the copies. The experts are spread out with expert parallelism, and an all-to-all sends each token to the accelerators that hold its experts. The same accelerators can act as separate data-parallel replicas in attention and as one expert-parallel group in the MoE layers.

According to their submitted configurations, the largest DeepSeek-V3 submissions all keep the expert all-to-all inside the scale-up domain:

  • NVIDIA (10,240 GB300 GPUs, 160 racks × 64 GPUs) [1]. Four pipeline stages, each interleaved into four virtual stages. The scale-out network carries pipeline and gradient traffic.
    • Attention layers: no tensor parallelism, which leaves 2,560 data-parallel replicas of 6 sequences each (global batch 15,360).
    • MoE layers: 32-GPU expert-parallel groups, each inside one NVL72 rack, with 8 experts per GPU and an all-to-all over NVLink. Each expert is replicated 80 times.
  • Azure (10,240 NVIDIA GB200 GPUs) [2]. The same pipeline layout. The Ethernet scale-out network carries pipeline and gradient traffic.
    • Attention layers: 2-way tensor parallelism to fit GB200’s 186 GB of HBM, which leaves 1,280 replicas of 12 sequences each.
    • MoE layers: the same 32-GPU expert groups over NVLink, with 80 replicas of each expert.
  • Google (8,192 TPU7x chips) [10]. One 8×32×32 ICI torus (16,384 TensorCores) with no pipeline stages. All traffic stays on ICI.
    • Attention layers: 8,192-way data parallelism across chips, with two sequences per chip (global batch 16,384). Each chip’s two TensorCores split the attention heads 2 ways, the equivalent of 2-way tensor parallelism.
    • MoE layers: the experts are split 16 ways across one torus axis and both TensorCores, with an all-to-all inside each 16-core group. That leaves 1,024-way data parallelism across the other two axes, over which the expert weights are also sharded.

All three overlap the expert all-to-all communication with computation. The GB200/GB300 jobs do it within the pipeline schedule, and the TPU job splits each MoE layer’s batch in half so one half’s attention overlaps the other’s all-to-all and expert computation.

On 8-GPU servers, Cisco [11] and the University of Florida [16] apply the same principle with a smaller domain: 8-way expert parallelism within a node, and 8–16 pipeline stages across nodes.

The other benchmarks call for different strategies, shaped mainly by model size, model type and global batch size:

  • Qwen3.5-397B GRPO (reinforcement learning) splits the system into a training pool and a generation pool. NVIDIA’s NeMo RL runs train the policy with TP4, PP2 and EP16–32 in MXFP8, keeping each expert-parallel group inside one NVLink domain, and generate rollouts asynchronously with TensorRT-LLM (with separate prefill and decode engines) or vLLM. As these systems grow from 64 to 1,024 GPUs, the split shifts toward generation, from 32 training and 32 generation GPUs to 256 and 768, and generation tensor parallelism rises from 4 to 8. Google pairs a 128-chip TPU7x trainer slice (FSDP 32, CP 4, EP 2) with 128 four-chip rollout slices running vLLM, broadcasts updated weights over the data-center network while allowing rollouts to lag the policy by up to two versions, and runs up to 4,096 CPU sandboxes for the agents’ tool calls and tests.
  • Llama 3.1 8B (dense pretraining) fits on a single accelerator, so up to 64–72 accelerators submitters use plain data parallelism with multiple sequences per accelerator. Beyond that the global batch (about 64 sequences) runs out: NVIDIA’s 512-GPU VR200 runs add TP2 and CP4, giving 64 data-parallel groups of eight GPUs with one sequence each, and the University of Florida’s 256-GPU B200 run uses TP4.
  • Llama 2 70B LoRA (fine-tuning) trains only small adapter weights, so gradient traffic is light, but the batch is tiny: 8 sequences at 8 accelerators. Submitters scale it by splitting each sequence across accelerators, from CP2–CP4 on B200 and B300 servers to CP8 on VR200 racks at 72 and 512 GPUs.
  • GPT-OSS-20B (MoE) runs with plain data parallelism on 8 accelerators. Multi-node runs add expert parallelism (EP4) and context parallelism (CP2–CP4), and NVIDIA’s 512-GPU runs combine TP2, CP4 and EP8.
  • FLUX.1 (text-to-image) uses pure data parallelism at every scale. Submitters grow the global batch from 512 to 2,048–4,096 images while the per-accelerator batch shrinks from 64 to as few as 4.
  • DLRMv4 (recommendation) shards its large embedding tables across accelerators with TorchRec model parallelism and runs the HSTU attention layers data-parallel, with no tensor, pipeline or context parallelism. The global batch grows from 8,192 to 32,768 samples as systems scale.

The pattern across benchmarks is consistent: the smaller the global batch relative to the system, the more submitters rely on context and tensor parallelism; MoE models add expert parallelism kept within the scale-up domain; and RL adds an inference pool that grows faster than the trainer.

Optimizations: faster on the same hardware

Comparing the best v6.1 result with the best v6.0 result for the same accelerator family at the same scale largely holds hardware constant, so the differences mostly reflect software, recipes and tuning, along with changes in which submitters ran each configuration. On the four benchmarks that are comparable across rounds, these gains are uneven: some configurations improved by 16–85%, while mature recipes were essentially unchanged. These ratios are MLCommons-derived, not official MLPerf results.

Same accelerator, same scale: v6.1 vs v6.0 at 8 accelerators

MLPerf Training v6.1

Closed, Available, RCP-passing. v6.0 best ÷ v6.1 best. GH200 excluded (identical results).

Show exact chart data

Source: MLCommons historical results database and final MLPerf Training v6.1 results. Power/reference rows excluded. No external data requests.

  • AMD software stack: MI350X and MI355X improved 1.16–1.19x on Llama 2 70B LoRA and Llama 3.1 8B at 8 GPUs [17–21], which AMD attributes to continued ROCm software optimization.
  • FLUX.1 on GB300: 1.51–1.85x faster at 16, 32, 72 and 512 GPUs [7, 22–28].
  • Driver-level work: tinycorp’s independent tinygrad stack, with a user-space GPU driver, nearly halved its Llama 3.1 8B time on MI350X (187 to 101 minutes) [29, 30].
  • Submitter-led optimization: the best 8-GPU GB300 results for Llama 3.1 8B (1.21x) and GPT-OSS-20B (1.40x), and the MI300X LoRA result (1.25x) [31–35], came from Firmus and KRAI, described below.
  • Mature recipes: best results on B200, B300 and H200, and Llama 2 70B LoRA on GB300, were within about 6% of v6.0, suggesting those recipes were already well optimized.

“In MLPerf Training 6.1, AMD delivers an average 19% increase in Llama training performance over round 6.0 on the same eight-GPU AMD Instinct™ MI350X and MI355X platforms.” — AMD

“This submission leveraged tinygrad with its python-based userspace driver for the MI350X, which delivered a performance uplift over the kernel driver through reduced overhead and more control over the hardware.” — tiny corp

These gains come from different submitters and comparisons, so they should not be combined into a single speedup. Together, they show that kernel design, memory use and the rest of the software stack continue to change the performance of existing platforms.

Numerical precision

Submitters now disclose the lowest numerical precision used in linear layers, attention and communication. In v6.1, 116 of 250 results used 4-bit formats in linear layers (NVFP4 on 82 results on NVIDIA accelerators and MXFP4 on 34 results on AMD accelerators), 113 used FP8, and the rest used MXFP6 or BF16.

The numerical precision strongly depends on the underlying workload. Every DeepSeek-V3 and Qwen3.5 GRPO result used FP8 in linear layers, and GPT-OSS-20B mostly did too (28 of 40). 4-bit formats were most common on FLUX.1 (24 of 27), Llama 3.1 8B (42 of 56) and Llama 2 70B LoRA (38 of 58), while DLRMv4 used MXFP6 or BF16. Attention and communication stay at higher precision: only one result used a 4-bit format in attention, and 189 of 250 use BF16 as their lowest communication precision.

AI-assisted optimization

Two submitters reported in their supplemental statements that they used AI agents and AI-generated code to improve on the vendor reference recipes they started from.

  • KRAI applied agentic engineering to Llama 2 70B LoRA fine-tuning on a Dell PowerEdge XE9680 with eight AMD Instinct MI300X GPUs. Starting from AMD’s optimized implementation, it cut time-to-train from 28.65 minutes in v6.0 to 22.83 minutes [34, 35], a 20% improvement on long-deployed hardware, with the help of Claude Opus 5.5 agents. Much of the gain came from avoiding work rather than computing faster: keeping all activations in memory removes the repeated forward computation during backpropagation. KRAI also fused the LoRA adapter computation into the base GEMM kernel.
  • Firmus built optimized GPU kernels on top of NVIDIA’s reference recipes on eight GB300 GPUs, covering attention, normalization, MLP, MoE, low-precision quantization, communication and optimizer operations. NVFP4-optimized kernels cut GPT-OSS-20B from 69.60 to 52.90 minutes (24%) and Llama 3.1 8B from 60.88 to 52.40 minutes (14%) [32, 33, 36, 37], compared with its own runs of the unmodified reference recipe using the same accelerator type and count.

“By storing activations in FP8 and packing dropout masks, we freed enough GPU memory to keep all activations, eliminating recomputation entirely. This alone cut the training step time by 12%.” — KRAI

“Firmus engineers combined AI-driven kernel generation with automated performance testing and mathematical validation, enabling rapid evaluation of alternative implementations.” — Firmus

Together these suggest that AI-assisted development is becoming a practical tool for finding performance headroom, even on hardware that has been in service for years.

Looking ahead

MLPerf Training v6.1 extends the suite to agentic RL and generative recommendation. The submissions in this round are more distributed, more varied in hardware and fabric, and more dependent on software optimization than in v6.0. Open questions for future rounds:

  • Scaling the benchmarks beyond the data-parallel limit set by their global batch size.
  • The importance of scale-out fabric as we introduce newer and larger models to our benchmark suite.
  • Increased competition among submitters driving further optimization.

We are looking forward to MLPerf Training v7.0 next year, where we anticipate new benchmarks and more innovation in systems and optimization.

Explore the results

All MLPerf Training v6.1 results are available in the interactive results explorer. The full supplemental discussion, including submitter-reported system details, is available in the v6.1 Supplemental PDF.

We invite submitters and other stakeholders to join the MLPerf Training working group and help evolve the benchmark.

Explore all results | Read the Supplemental | Join the Working Group

Results cited

All results are MLPerf® Training, Closed division, as published by MLCommons. Times are official time-to-train scores in minutes. Ratios in the text are MLCommons-derived from these results.

[1] NVIDIA, Theia-jhb (160x NVIDIA GB300 NVL72), 10,240 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available Cloud. DeepSeek-V3 671B: 1.38 min. Result ID 6.1-0029.

[2] Azure, Azure GB200 (160x NVIDIA GB200 NVL72), 10,240 × NVIDIA Blackwell GPU (GB200). MLPerf® Training v6.1, Closed, Available Cloud. DeepSeek-V3 671B: 2.36 min. Result ID 6.1-0002.

[3] NVIDIA, Hecate (8x NVIDIA VR NVL72), 512 × NVIDIA Rubin GPU. MLPerf® Training v6.1, Closed, Available on-premise. Llama 3.1 8B: 3.95 min; Llama 2 70B LoRA: 0.313 min; GPT-OSS-20B: 5.48 min. Result ID 6.1-0133.

[4] NVIDIA, Hecate (8x NVIDIA VR NVL72), 512 × NVIDIA Rubin GPU. MLPerf® Training v6.1, Closed, Available on-premise. FLUX.1: 6.88 min. Result ID 6.1-0134.

[5] NVIDIA, Tyche-hsg (16x NVIDIA GB200 NVL72), 1,024 × NVIDIA Blackwell GPU (GB200). MLPerf® Training v6.0, Closed, Available Cloud. Llama 3.1 8B: 4.46 min. Result ID 6.0-0015.

[6] NVIDIA, Theia (8x NVIDIA GB300 NVL72), 512 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.0, Closed, Available on-premise. Llama 2 70B LoRA: 0.400 min; GPT-OSS-20B: 7.43 min. Result ID 6.0-0101.

[7] NVIDIA, Theia (8x NVIDIA GB300 NVL72), 512 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.0, Closed, Available on-premise. FLUX.1: 17.12 min. Result ID 6.0-0100.

[8] NVIDIA, NVIDIA H200 (8x H200-SXM-141GB), 8 × NVIDIA H200-SXM5-141GB. MLPerf® Training v4.0, Closed, Available on-premise. Llama 2 70B LoRA: 24.69 min. Result ID 4.0-0072.

[9] NVIDIA, Hecate (1x NVIDIA VR NVL72), 8 × NVIDIA Rubin GPU. MLPerf® Training v6.1, Closed, Available on-premise. Llama 2 70B LoRA: 2.76 min. Result ID 6.1-0125.

[10] Google, tpu7x-8x32x32, 8,192 × TPU7x (Ironwood). MLPerf® Training v6.1, Closed, Available Cloud. DeepSeek-V3 671B: 1.70 min. Result ID 6.1-0026.

[11] Cisco, Zeus_8xB300_SXM_288GB_multi_planar, 64 × NVIDIA Blackwell Ultra GPU (B300-SXM-270GB). MLPerf® Training v6.1, Closed, Available on-premise. DeepSeek-V3 671B: 163.19 min. Result ID 6.1-0069.

[12] Crusoe, 16xMI355X-288GB_2xEPYC-9575F, 128 × AMD Instinct MI355X 288GB HBM3e. MLPerf® Training v6.1, Closed, Available Cloud. Llama 3.1 8B: 14.07 min. Result ID 6.1-0003.

[13] NVIDIA, Theia-jhb (4x NVIDIA GB300 NVL72), 256 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available Cloud. DeepSeek-V3 671B: 31.40 min. Result ID 6.1-0039.

[14] NVIDIA, Theia-jhb (16x NVIDIA GB300 NVL72), 1,024 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available Cloud. DeepSeek-V3 671B: 8.03 min. Result ID 6.1-0030.

[15] NVIDIA, Hecate (1x NVIDIA VR NVL72), 72 × NVIDIA Rubin GPU. MLPerf® Training v6.1, Closed, Available on-premise. Llama 3.1 8B: 7.38 min. Result ID 6.1-0123.

[16] University of Florida, HiPerGator NVIDIA DGX B200 (8x B200-SXM-180GB), 256 × NVIDIA Blackwell GPU (B200-SXM-180GB). MLPerf® Training v6.1, Closed, Available on-premise. DeepSeek-V3 671B: 132.16 min. Result ID 6.1-0148.

[17] MiTAC, MI350X_akash_1node_primus, 8 × AMD Instinct MI350X 288GB HBM3e. MLPerf® Training v6.0, Closed, Available on-premise. Llama 2 70B LoRA: 10.09 min. Result ID 6.0-0094.

[18] MiTAC, G8825Z5, 8 × AMD Instinct MI350X 288GB HBM3e. MLPerf® Training v6.0, Closed, Available on-premise. Llama 3.1 8B: 108.97 min. Result ID 6.0-0092.

[19] AMD, Supermicro AS-8126GS-TNMR, 8 × AMD Instinct MI350X 288GB HBM3e. MLPerf® Training v6.1, Closed, Available on-premise. Llama 2 70B LoRA: 8.72 min; Llama 3.1 8B: 92.08 min. Result ID 6.1-0067.

[20] AMD, smci355-ccs-aus-n12-05 (AS -4126GS-NMR-LCC), 8 × AMD Instinct MI355X 288GB HBM3e. MLPerf® Training v6.0, Closed, Available on-premise. Llama 2 70B LoRA: 8.27 min; Llama 3.1 8B: 86.84 min. Result ID 6.0-0036.

[21] AMD, Supermicro AS-4126GS-NMR-LCC, 8 × AMD Instinct MI355X 288GB HBM3e. MLPerf® Training v6.1, Closed, Available on-premise. Llama 2 70B LoRA: 6.98 min; Llama 3.1 8B: 72.94 min. Result ID 6.1-0062.

[22] Dell, 4xXE9712x4GB300, 16 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.0, Closed, Available on-premise. FLUX.1: 112.45 min. Result ID 6.0-0059.

[23] Quanta Cloud Technology, D75U-1U_ngpu16, 16 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available on-premise. FLUX.1: 72.40 min. Result ID 6.1-0136.

[24] Oracle, BM.GPU.GB300.4, 32 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.0, Closed, Available Cloud. FLUX.1: 65.03 min. Result ID 6.0-0029.

[25] Dell, 8xXE9712x4GB300, 32 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available on-premise. FLUX.1: 42.99 min. Result ID 6.1-0090.

[26] GigaComputing, DLB2-CB3, 72 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.0, Closed, Available on-premise. FLUX.1: 36.54 min. Result ID 6.0-0063.

[27] Quanta Cloud Technology, D75U-1U_ngpu72, 72 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available on-premise. FLUX.1: 23.46 min. Result ID 6.1-0138.

[28] NVIDIA, Theia-jhb (8x NVIDIA GB300 NVL72), 512 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available Cloud. FLUX.1: 9.23 min. Result ID 6.1-0044.

[29] tinycorp, tinybox 8xMI350X, 8 × AMD Instinct MI350X 288GB HBM3e. MLPerf® Training v6.0, Closed, Available on-premise. Llama 3.1 8B: 187.48 min. Result ID 6.0-0115.

[30] tinycorp, tinybox 8xMI350X, 8 × AMD Instinct MI350X 288GB HBM3e. MLPerf® Training v6.1, Closed, Available on-premise. Llama 3.1 8B: 101.28 min. Result ID 6.1-0153.

[31] HPE, NVIDIA GB300 NVL72 by HPE, 8 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.0, Closed, Available on-premise. Llama 3.1 8B: 63.52 min; GPT-OSS-20B: 74.05 min. Result ID 6.0-0078.

[32] Firmus, Firmus Supermicro GB300 NVL72, 8 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available on-premise. Llama 3.1 8B: 52.40 min. Result ID 6.1-0094.

[33] Firmus, Firmus Supermicro GB300 NVL72, 8 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available on-premise. GPT-OSS-20B: 52.90 min. Result ID 6.1-0091.

[34] KRAI, Dell PowerEdge XE9680 (8x MI300X-192GB), 8 × AMD Instinct MI300X 192GB HBM3. MLPerf® Training v6.0, Closed, Available on-premise. Llama 2 70B LoRA: 28.65 min. Result ID 6.0-0081.

[35] KRAI, PowerEdge_XE9680_8xMI300X, 8 × AMD Instinct MI300X 192GB HBM3. MLPerf® Training v6.1, Closed, Available on-premise. Llama 2 70B LoRA: 22.83 min. Result ID 6.1-0113.

[36] Firmus, Firmus Supermicro GB300 NVL72, 8 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available on-premise. GPT-OSS-20B: 69.60 min. Result ID 6.1-0092.

[37] Firmus, Firmus Supermicro GB300 NVL72, 8 × NVIDIA Blackwell Ultra GPU (GB300). MLPerf® Training v6.1, Closed, Available on-premise. Llama 3.1 8B: 60.88 min. Result ID 6.1-0093.

MLPerf® and MLCommons® are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.