On October 19, 2026, OpenAI reported results from two distinct coding evaluations: MultiLangBench tests whether models can build SQLite-compatible database engines in 20 programming languages, while a separate evaluation tests COBOL migrations that preserve observable behavior. OpenAI reported higher scores for newer models on both tasks and said it was releasing the MultiLangBench setup, test suite, and language-level results on GitHub.
NeoTeo previously covered GPT-6 Astra’s broader capabilities. These evaluations focus on narrower coding tasks, with different scoring methods.
MultiLangBench checks database behavior across languages
OpenAI describes MultiLangBench as a test of building a SQLite-compatible database engine from scratch in each of 20 languages. A model receives an initial database and an SQL query, then returns a modified database. Reference SQLite checks whether that database contains the expected results. OpenAI says the tests were adapted from SQLite’s test suite.
At maximum reasoning effort, OpenAI reported an average pass rate of 87.4% for GPT-6 Astra, compared with 26.8% for GPT-5.6 Sol. The difference between those reported averages is 60.6 percentage points. OpenAI said GPT-6 Astra scored above 83% in 19 of the 20 languages; its reported COBOL score was 72.1%, the lowest of the named examples.
OpenAI reported a 40.3% average pass rate for GPT-6 Luna at maximum reasoning effort. It estimated the evaluation’s average API cost at $0.77 per sample for GPT-6 Luna and $3.48 for GPT-5.6 Sol. Those figures describe costs in this evaluation, not general API rates.
COBOL migration uses a different success test
OpenAI’s separate evaluation measures whether a COBOL program can be migrated to C++, Java, Kotlin, Python, or Rust while preserving its behavior. Hidden tests check for byte-for-byte parity in observable outputs, and every test must pass for a migration to count as successful.
| Model | Reasoning effort | Reported accuracy |
| GPT-6.1 Sol | Low | 86.6% |
| GPT-6.1 Sol | Higher | 97.9% |
| GPT-5.6 Sol | Low | 5.4% |
At low reasoning effort, the reported scores differ by 81.2 percentage points. OpenAI also said GPT-6.1 Sol’s API cost on this evaluation was 61% of GPT-5.6 Sol’s, or 39% lower. The accuracy figures measure a different task from MultiLangBench’s database-engine pass rates, so the two sets of scores are not interchangeable.
A creator’s custom test used different tasks
In a video published September 6, 2026, Povilas of AI Coding Daily reported that GPT-6 Astra scored 20 out of 20 across four custom coding projects. He also described manually adjusting Astra’s leaderboard score after automated code-quality judges penalized choices such as using strings instead of enums. His projects were separate from both OpenAI evaluations and do not reproduce their tests.
Production work still needs expert review
OpenAI said production use requires expert review and testing against the relevant business rules and dependencies. MultiLangBench checks database outputs, and the COBOL evaluation checks observable program outputs; applying either result to a live system still calls for tests specific to that system’s requirements.
