On October 29, 2026, Tom’s Hardware described a prototype agentic AI CPU benchmark that counts completed agent tasks per second as concurrent workers increase. Its methodology report presents a test design, not comparative CPU scores.
In this article
- What the benchmark counts
- How recorded sessions are replayed
- Frozen replay and live inference measure different things
- Workloads and systems in the prototype
- Why other agent-CPU claims use different yardsticks
What the benchmark counts
A session is one agent task completed from start to finish. The proposed throughput metric is sessions per second, recorded at each concurrency level. Peak throughput is the highest observed rate, along with the number of concurrent sessions at which it occurred.
Tom’s Hardware also defines an “ideal throughput” point using response time: the slowest 5% of sessions must take no more than 1.5 times the duration of a single session at base concurrency. That threshold pairs a throughput rate with a limit on how much the slowest sessions can stretch as more agents run.
How recorded sessions are replayed
For the recording phase, a proxy forwards model requests to a live llama.cpp server and saves the requests and replies. Tool calls and their results are logged separately. During replay, each model request is matched by session label and hash; a mismatch invalidates the run. The test also compares tool calls with the recorded calls and returns recorded tool results to keep tasks on the same path.
The concurrency sweep starts with one worker, increases to the CPU’s maximum thread count, and repeats each level three times. Workers start at different points in a sequence of 30 tasks, so tasks overlap and take different amounts of time.
Frozen replay and live inference measure different things
In the frozen replay sweep, agents use recorded model replies, so live GPU inference is excluded. A separate live-inference run generates the same number of tokens as the recording, discards the generated reply, and returns the stored one to the agent. That run includes model inference, prefill and decode work, cache behavior, and agent queueing.
| Run | Model response | Live inference | What it captures |
| Frozen replay | Uses a recorded reply | Excluded | Concurrent agent work and tool workflows while replayed responses keep tasks consistent |
| Live-inference run | Generates matching tokens, then returns the recorded reply | Included | Inference behavior, cache effects, agent queueing, and tool workflows |
That separation matters when interpreting what a score would represent: replay focuses on agent-side work under repeatable responses, while the live run includes the inference pipeline and its effect on queued agents.
Zen van Riel’s separate demonstration uses Claude Code hooks and Python scripts to track CPU activity during agent sessions. He describes it as an illustrative exercise rather than a rigorous test, so it shows an instrumentation approach rather than a standardized CPU comparison.
Workloads and systems in the prototype
The prototype uses custom Python agents built with LangGraph and six personas: analyst, coding, document processing, research, support, and web. The tasks are currently single-turn, though an individual task can involve multiple model calls and tool use. The research persona searches a local English-language Wikipedia corpus of about 50,000 articles; the coding persona works with four small projects, two in C and two in Python.
Tom’s Hardware describes two system types: a 64-core Threadripper test bed with 128 GB of DDR5-6000 RDIMMs, and an NVIDIA DGX Spark with a GB10, 128 GB of unified memory, a Blackwell GPU, and an Arm Grace CPU. The model used to record traces was Gemma 4 26B A4B on the DGX Spark. Tom’s Hardware says CPU load can vary with GPU performance, model, quantization, and server settings, so a replay result answers a narrower question than the CPU demand of every live agent setup.
Agent workflows can involve orchestration, retrieval, code compilation, and sandbox activity. Tom’s Hardware notes that CPU work may be a small part of one sequential session, while simultaneous sessions can compete for CPU resources. For its current setup, it reports Docker isolation overhead below 2% compared with bare metal.
Why other agent-CPU claims use different yardsticks
Published rack and sandbox figures measure different workloads and boundaries from sessions per second in the proposed test. Signal65, in an analysis commissioned by Arm, reports average modeled rack performance of 1.99× for Arm AGI CPU versus AMD EPYC 9755 across seven workloads. AMD reports a modeled EPYC 9965 result of 2.37× the normalized rack-level geometric mean of NVIDIA Vera in a 100 kW, two-socket infrastructure comparison; AMD says that workload set does not model a full end-to-end agent pipeline. NVIDIA reports more than 1.8× sandbox performance for Vera versus traditional x86 architectures under full load. None of those figures is a result from Tom’s Hardware’s sessions-per-second sweep.
A separate open-source project, ritikraj7/cpu-centric-agentic-ai, describes workloads for web search, retrieval-augmented generation, code generation, math problem-solving, and chemistry research. Its README lists CUDA 11.8 or later for GPU acceleration and at least 16 GB of GPU memory for LLM inference, giving that suite a different hardware scope from a frozen replay focused on CPU-side concurrency.
