Memdoor reports 49 successful problem solves in 50 attempts using Qwen3.8-27B in its team-solve workflow: five passes over ten LiveCodeBench Hard problems. The Fable 5 score in the project’s results is a separate one-call comparison; Memdoor did not run Fable 5 with team-solve. Memdoor’s team-solve repository lists the scores for each pass.
How the five-pass scores compare
The results cover the first ten problems in the GVS5H authors’ LiveCodeBench Hard list, with each configuration scored over five passes. The repository reports these pass scores and means:
| Method and configuration | Problems solved per pass, out of 10 | Mean per pass, out of 10 |
Memdoor team-solve, Qwen3.8-27B | 10, 10, 10, 10, 9 | 9.8 |
| GVS5H authors’ harness, Qwen3.8-27B | 9, 10, 10, 9, 10 | 9.6 |
| Fable 5, one call | 7, 8, 10, 8, 7 | 8.0 |
Memdoor describes its 9.8-versus-9.6 mean against the GVS5H authors’ harness as within noise. For that harness comparison, the repository reports Qwen3.8-27B, the authors’ prompts and loop, a 128K reply cap, and grading against hidden tests with a corrected grader. The Fable 5 column uses a different setup, so the table does not show how Fable 5 would perform in team-solve.
What the workflow does
team-solve assigns six agent roles: a planner, two ideators, a solver, a reviewer/manager, and a repair agent. The two ideators run in parallel; the remaining roles follow the workflow’s dependencies. Testing and final verification are commands, not agent roles. The steps use a user-selected model pin, and agents pass work through files with fresh contexts. Six is the number of listed roles, not a stated count of required GPUs or concurrently running model instances.
The benchmark is a contest-style coding task: each problem has a written prompt and sample input/output, and the answer is a Python program that reads from standard input. Memdoor reports that its runs were hosted on OpenRouter, with a median duration of 40 minutes and a median cost of $0.39 per run. It reports $35.79 in total costs for the 50 runs.
What the result says about the model comparison
These figures describe a ten-problem sample, not general coding accuracy. The Fable 5 result is a one-call baseline from separate files, while Memdoor’s workflow adds planning, review, and repair. The results therefore compare the reported configurations, not the two models under the same agent workflow.
The repository describes one failed problem in which the manager generated an incorrect counterexample and the workflow rewrote a correct program. It also records two other cases where correct programs were overwritten and later recovered. Memdoor notes that the contests date to spring 2025 and that possible contamination cannot be ruled out.
