Kingy’s October 23, 2026, test of Kimi K3.1 Pro found that four of eight patches passed the required tests, but none of the eight attempts completed a normal submission. A separate one-shot demonstration by Beau Johnson reported successful results on two short generation tasks and a perfect bug-hunt score, but the tests measure different things and neither establishes a general coding ranking.
Passing tests did not complete the workflow
A passing patch means the saved code passed the required tests. Kingy counted a task as normally submitted only when the agent also printed COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT and provided a final explanation. Kimi K3.1 Pro left four passing patches, but did not complete that handoff in any of its eight attempts.
Results from four selected repository issues
Kingy ran two attempts per model on each of four historical issues, from Django, pytest, scikit-learn, and Sphinx. That makes eight attempts per model, but only four distinct issues. The runner used mini-SWE-agent 2.4.6 in fresh Linux containers with one CPU, 2 GiB of memory, and no network access.
| Model | Patches passing required tests (of 8) | Normal submissions (of 8) |
| Kimi K3.1 Pro | 4/8 | 0/8 |
| GPT-6.1 Sol | 5/8 | 0/8 |
| Claude Sonnet 5.5 | 4/8 | 2/8 |
| Kimi K3 | 0/8 | 0/8 |
| DeepSeek V4.1 Flash 1017 | 0/8 | 0/8 |
Kimi K3.1 Pro passed both attempts on the pytest and scikit-learn issues, and produced no patch in its Django and Sphinx attempts. The table separates test-passing patches from completed submissions because they were different outcomes in this run.
Why the Kimi attempts stopped
Kingy attributed the four Kimi runs with passing patches stopping to its conservative input-reservation check. In the first pytest attempt, the run recorded nine model calls, 87,885 cumulative input tokens, 3,655 output tokens including reasoning, about 87 seconds, and $0.083296 in metered API use. Across all eight Kimi coding attempts, including failed attempts, Kingy reported $0.722 in API use.
What a separate one-shot test found
In Beau Johnson’s video, Kimi K3.1 Pro completed a Minecraft-generation task in 4 minutes 28 seconds for $0.15, and a low-poly horse task in 4 minutes 39 seconds for $0.15. Kimi K3.1 Pro, Claude Opus 5.5, and Claude Sonnet 5.5 each scored 10/10 on a bug hunt; the video says Claude scored the answers. Johnson ran each task once, so these results describe those demonstrations rather than repeated performance.
Different benchmarks measure different tasks
Moonshot AI separately reported a 65.7% score for Kimi K3.1 Pro on Terminal-Bench 4.0, compared with 66.4% for Claude Opus 5.5 and 58.2% for GPT-6 Astra. Those company-reported benchmark results use a different evaluation from Kingy’s selected repository issues and Johnson’s one-shot tasks. The results describe those specific tests; they do not establish an overall coding-model ranking.
