HarnessMM’s authors report substantially fewer tokens per task in specific coding benchmarks, but the reduction varies by harness and test. In a 98-task HumanEval comparison, DeepSeek Harness used 96.28 thousand tokens per task without the method and 11.62 thousand with it; reported success rose from 71.43% to 85.71%. Those are benchmark results, not measurements of API bills or runtime. The authors’ paper, version 1 dated October 23, 2026, describes the method and its tests.
How HarnessMM reuses resolved actions
A coding-agent harness coordinates the model’s interaction with tools. HarnessMM changes that coordination layer; the paper says the model’s weights remain fixed. It stores actions the reasoning model has already resolved, then checks whether later tool feedback satisfies the conditions for reusing one.
The method represents stored actions in Executable Decision Graphs, which contain tool arguments, dependencies, and guards. Guarded Decision Reuse checks those guards against observed tool receipts. When the conditions are met, Certified Callback Substitution runs the eligible action through the harness’s native tool interface and records that the usual reasoning callback was omitted. If a dependency is unmet, the state is unsupported, or a new semantic decision is needed, control returns to the original reasoning actor.
The certificate records the execution route and omitted callback. It does not prove that the stored action was semantically correct or establish what the skipped model call would have said.
Results differ across benchmarks and harnesses
The authors compared input-plus-output tokens per task and task success. The table shows the integrated benchmark comparisons, with each result tied to its test set.
| Benchmark and harness | Test tasks | Tokens per task without / with HarnessMM (thousands) | Success without / with HarnessMM |
| HumanEval, DeepSeek Harness | 98 | 96.28 / 11.62 | 71.43% / 85.71% |
| HumanEval, Codex | 98 | 66.72 / 47.02 | 90.82% / 90.82% |
| MBPP-sanitized, DeepSeek Harness | 257 | 53.98 / 10.61 | 90.27% / 91.83% |
| MBPP-sanitized, Codex | 257 | 129.55 / 49.22 | 91.83% / 90.66% |
The largest token reduction in these pairs is DeepSeek Harness on HumanEval. Codex’s HumanEval success rate was unchanged, while its MBPP-sanitized success rate fell from 91.83% to 90.66%. The DeepSeek Harness MBPP-sanitized comparison went the other way, with reported success increasing from 90.27% to 91.83%. The results therefore vary with both the harness and benchmark.
What the token and call counts include
The paper uses custom 20/20/60 training, validation, and test splits: 33/33/98 tasks for HumanEval and 85/85/257 for MBPP. Its token totals count model input and output across paid calls for each task, including failed calls, fallbacks, and cached input. A token total is not a dollar amount or a measure of elapsed time.
For the 98-task HumanEval DeepSeek Harness comparison, the paper’s audit records 954 paid model calls without HarnessMM and 415 with it. It separately records 147 certified callback bypasses. The call totals and certified bypasses are different measures; the 539-call difference is not a count of certified bypasses.
A separate MBPP test isolates method components
The paper also reports a 257-task MBPP mechanism ablation, separate from the integrated DeepSeek Harness comparison above. In that test, full HarnessMM used 13.17 thousand tokens per task with 93.77% success. Single-use graphs used 18.03 thousand tokens with 93.00% success; retaining callbacks used 16.37 thousand with 93.39% success; and disabling graph execution used 16.19 thousand with 95.33% success.
These figures come from different ablation conditions than the integrated MBPP-sanitized result of 10.61 thousand tokens per task and 91.83% success. The ablation also shows that the lowest token count and highest success rate did not occur in the same condition. The authors note that new semantic decisions still require reasoning, and that adapters are task-specific.
