Google’s TPUv7 benchmarks depend on model and setup

On November 5, 2026, SemiAnalysis reported four Google TPUv7 submissions to AgentX, a long-context, multi-turn inference benchmark. The results differed by model, numerical precision, serving software and measurement: they describe particular configurations, not one overall ranking of AI accelerators.

A benchmark result is useful when its workload, precision and metric match the question you need answered.

What AgentX measures

AgentX tests multi-turn AI-agent sessions with long contexts, substantial reuse of earlier prompt text, and bursts of work by sub-agents, according to SemiAnalysis. That makes it relevant to this kind of agentic inference serving, but not a proxy for every coding, research or office workload.

Two metrics in the reported comparisons answer different questions. Tokens per dollar measures how much output a serving setup delivers for its cost; it is not a measure of speed by itself. Decode is the stage where a model generates output tokens. End-to-end performance includes the broader request, including processing the input before generation. A system can lead on one and trail on another.

FP8 and FP4 refer to different numerical precisions used for model weights. The precision can affect a comparison, and SemiAnalysis notes a quality trade-off in its FP4 comparisons. Keep that in mind when reading results across different formats.

Four model setups, with different results

SemiAnalysis reported results for Muse Spark Open, GLM-5.2, Qwen3.5 397B and Kimi K3. Each used a distinct setup, so the figures below belong to their stated model and test conditions.

For Muse Spark Open, RadixArk submitted an SGLang-TorchTPU configuration using NVFP4 weights. SemiAnalysis described TPUv7 as competitive on decode performance per dollar using Meta’s reported TPU cost basis, while saying end-to-end latency still needed work, including prefill-kernel optimization. That is a mixed result, not a single performance verdict.

For GLM-5.2 in FP8, SemiAnalysis reported TPUv7 delivering 1.2–2.7 times the tokens per dollar of B200, B300 and MI355X across targets of 60–300 tokens per second per user. At 150 tokens per second per user, it reported 4.4 times the tokens per dollar of H200.

The GLM-5.3 FP4 comparison favored NVIDIA systems at the stated operating point. At 100 tokens per second per user, SemiAnalysis reported GB300 and GB200 FP4 at 3.6 and 3.5 times TPUv7’s tokens per dollar, respectively. The analysis attributed the difference in part to TPUv7’s lack of native FP4 support and noted the precision trade-off.

For Qwen3.5 397B, the initial comparison put TPUv7 FP8 with vLLM at 1.67 times the performance per dollar of GB300 NVL72 FP4. A later NVIDIA FP4 submission reportedly led on decode, while TPUv7 remained ahead on end-to-end performance in the tested setup. Those findings can coexist because the measured stages differ. The llm-d setup used 24–32 TPUv7 chips per result, with four chips per host. At each system’s fastest tested point, SemiAnalysis reported a p90 time to first token of 8.6 seconds for TPUv7 and 11.1 seconds for GB300 FP4.

The Kimi K3 results changed across submissions. In the launch-period comparison, SemiAnalysis reported TPUv7 at 0.77–0.91 times GB300’s tokens per dollar across targets of 50–200 tokens per second per user. In a single-user comparison, it reported 391 tokens per second per user for TPUv7 and 251 for GB300. After four NVIDIA refreshes, the analysis reported GB300 FP4 leading TPUv7 by 1.5–2.6 times on tokens per dollar. Its later comparison also put TPUv7 at 1.15–2.5 times the tokens per dollar of GB200, B200 and B300. These figures describe different submission states and comparisons.

Why a single winner misses the point

The GLM and Qwen results show how much the answer depends on the chosen measure. TPUv7’s reported GLM-5.2 FP8 cost advantage applies to the specified 60–300-token-per-second-per-user range and named competitors. The GLM-5.3 FP4 comparison, at 100 tokens per second per user, went the other way. For Qwen3.5, the reported decode and end-to-end leaders differed. A headline that reduces those comparisons to “TPU beats GPU,” or the reverse, would erase the conditions that produced the result.

The Kimi comparison adds another variable: submissions were refreshed. Its launch-period and post-refresh figures reflect different states of the benchmark, not one unchanged result. For anyone comparing accelerator claims, matching the model, precision, serving stack, operating target and metric is more useful than comparing a lone multiplier.

Google Cloud’s Ironwood inference examples

Google Cloud lists Ironwood inference examples for serving GPT OSS-120B with vLLM on GKE clusters and Qwen3-Coder-480B with vLLM, alongside PyTorch/TorchTPU inference examples in its Cloud TPU sample library. These are documented deployment examples for exploring inference workflows; they are not performance measurements for the AgentX comparisons.