NVIDIA’s 8-node DGX Spark benchmarks show workload-specific scaling

NVIDIA’s October 22, 2026 technical report puts its eight-node DGX Spark inference result at 52.58 decode tokens per second (TPS) for Qwen3.8-27B-NVFP4, compared with 11.86 TPS on one node. That is 4.43× the one-node decode throughput in this specific test. NVIDIA’s separate nanochat pretraining run reached 7.79× the one-node throughput, a result for training rather than inference.

Qwen inference results vary by metric and topology

NVIDIA measured Qwen3.8-27B in NVFP4 at concurrency one, with 8,192 input tokens and 256 output tokens; speculative decoding was disabled. Prefill is the processing of input tokens, while decode is the generation of output tokens. The table shows NVIDIA’s reported throughput for each configuration.

In this Qwen test, eight nodes delivered 4.43× the one-node decode throughput and 1.94× the prefill throughput. The gains differ by metric, and adding eight times as many nodes did not yield eight times the throughput. With eight nodes, NVIDIA reported 52.58 decode TPS for the switched RoCEnante setup, versus 40.11 TPS for the physical ring.

Ring, switch, RoCEnante, and NCCL

NVIDIA describes a switchless ring as a simpler, lower-cost option for four-node inference. Its guidance calls for a switch for eight-node layouts, independent groups within a cluster, or workloads that need direct all-to-all connectivity. A ring cannot represent two independent groups of four, and it can add hop penalties for some point-to-point workloads.

RoCEnante is a switch-only collective communication backend. For the Qwen test, NVIDIA reports decode scaling of 3.06× at four nodes and 4.43× at eight nodes with RoCEnante, compared with 2.56× and 3.05× without it. NVIDIA describes RoCEnante as suited to small messages, low concurrency, and a modest number of ranks. Its all-to-all exchange followed by local reduction reduces communication steps in that pattern, while data movement grows as message size or node count increases.

NVIDIA identifies NCCL as the supported default and says it can be a better fit for higher concurrency, larger clusters, or ring topologies. RoCEnante is merged into FlashInfer and falls back to NCCL if its startup test fails.

Switchyard routes requests to different models

NVIDIA’s Switchyard prototype sends simpler requests to MiniCPM5 2B and harder ones to Qwen3.8 Flash Next. In its eight-node configuration, three two-node Qwen replicas and two single-node MiniCPM replicas handled the requests.

NVIDIA reports that the routed setup matched its Qwen-only configuration at 91/100 on a 92-case tool-use benchmark. Aggregate throughput was about 1.5× when half of requests went to Qwen, and about 3.0× when one quarter went to Qwen and the rest to MiniCPM. Those figures belong to the prototype and its stated routing mixes.

nanochat pretraining reached 7.79× throughput

NVIDIA’s separate nanochat pretraining run used a global batch of 524,288 tokens per step and 32 sequences per GPU microbatch. The model had about 896.5 million parameters. NVIDIA reported these results:

The near-eightfold result applies to this pretraining workload and its batch configuration. It does not describe the Qwen inference test, where decode and prefill scaled differently.

One terabyte of memory accommodates a large checkpoint

NVIDIA specifies 128 GB of unified memory per DGX Spark, or 1 TB across eight nodes. The company reports running the full GLM 5.3 (753B) checkpoint, about 470 GB in NVFP4, across an eight-node cluster.

For one DGX Spark, NVIDIA lists memory bandwidth of 273 GB/s and a ConnectX-7 network interface rated at 200 Gbps. These are per-system specifications, not measured end-to-end cluster throughput.

In its October 22 report, NVIDIA said eight-node support in NVIDIA Sync was planned for release by the end of 2026.