Mozilla report puts Chinese open-weight AI models 4.4 months behind closed systems

Mozilla’s State of Open Source AI v1.1, published on September 15, 2026, estimates that leading open-weight AI models trailed closed frontier systems by approximately 4.4 months in a fitted comparison based on METR task-horizon data. That is a meaningful narrowing of the gap—but it is not a declaration of universal parity. The result changes with the benchmark, hardware, workload and deployment model.

For developers and companies, the practical choice is therefore less dramatic than “open beats closed” or “closed still wins.” Open weights can deliver lower hosted costs, portability and more control. Closed systems still offer an easier operational path and retain advantages in some demanding professional workloads.

Mozilla’s 4.4-month estimate comes with conditions

The 4.4-month figure describes the estimated difference between the lengths of tasks that leading open and closed systems can complete reliably, using a fitted analysis of METR task-horizon data. It is a measurement from Mozilla’s September 2026 analysis, not a live guarantee about every model or benchmark.

Mozilla places the main current advantage for closed systems in tasks that take roughly eight to twelve hours for a human expert. Tasks lasting less than eight hours are generally within reach of either category, while neither category is generally reliable beyond twelve hours in the cited analysis.

That framing matters. A model can look nearly equivalent on a coding or general intelligence benchmark and still be less useful for a long, complex professional assignment. Conversely, a model that falls slightly behind at the frontier may be entirely adequate for routine automation where cost, portability or local control matters more than the last few points of benchmark performance.

The same analysis also warns against treating one leaderboard as permanent. Mozilla’s cited Artificial Analysis snapshot was dated September 1, 2026, and used version 4.1.1. Later leaderboards or different evaluation sets are not automatically comparable.

Open weights are not the same as open source

An open-weight model makes its trained parameters downloadable. That can let an organization run the model independently, modify its deployment and choose among hosting options, subject to the model’s license.

Open source, in the stricter sense used by the Open Source Initiative, requires more. It involves access or transparency around source code and training data, as well as freedoms related to redistribution and derived works. A downloadable checkpoint alone does not make the training process reproducible.

Mozilla’s analysis lists 16 notable open releases and says none provides the complete data recipe required by its definition of open-source AI. In practice, many Chinese models are best described as open-weight: the weights are available, while some combination of the training data, data pipeline, code or full reproduction instructions remains unavailable.

That distinction has direct consequences. Open weights can improve deployment flexibility without providing complete visibility into how a model was trained or evaluated. For organizations handling sensitive decisions, that transparency gap can matter as much as the benchmark score.

Where the models come close—and where they do not

The September 1 snapshot placed four closed models at the top of the cited Artificial Analysis Intelligence Index, followed by four open-weight models. Kimi K3 and GLM-5.3 each scored 60 in the cited version 4.1.1 table. Kimi K3 was three points below the cited closed leader and two points below Claude Fable 5 in the same list-price comparison.

A separate neutral Terminal-Bench 2.1 comparison produced a similarly close but not identical picture:

The figures compare the same benchmark and task-price measure, but they do not turn one model into a universal winner. GLM-5.2 came close on this setup while costing less per completed task. Closed models retained the higher scores.

The gap also widens under hardware constraints. Mozilla reported a best open-model score of 52.6 on one eight-GPU server and 40 on a single B300 GPU, compared with 60 for the leading small-data-center entry. Hosted API results and self-hosted results answer different questions: one measures access to a service, while the other includes the physical limits of the deployment.

The real dividing line is the workload

Closed models remain most compelling when the work demands long-context fidelity, expert professional knowledge or high-intensity retrieval. They also reduce the operational burden: the provider supplies hosted access, while organizations can buy support, compliance packaging and accountability alongside the model service.

Open-weight systems are more attractive when a team needs portability, wants to choose its own host or can justify the engineering needed for deployment and maintenance. They can also make sense for routine workloads where a small performance difference is less important than predictable cost and control.

The decision does not have to be all-or-nothing. Mozilla’s analysis describes payment for closed models as workload-specific rather than organization-specific. A company might use an open-weight model for recurring, cost-sensitive tasks and reserve a closed frontier system for deadline-driven or unusually demanding work.

That is a more useful way to read the four-month estimate. It is not a promise that every open model will handle every job as well as a closed model. It is evidence that the category gap has become small enough for the workload—and not the label alone—to drive the decision.

Cheap API access is not cheap ownership

The hosted price comparison is striking. GLM-5.2’s cited neutral-harness task price was $0.43, compared with $1.98 for Claude Opus 4.7 and $2.41 for Claude Opus 4.8. But those are list-price or API-to-API observations, not complete cost-of-ownership calculations.

Running an open-weight model can require accelerators, serving infrastructure, engineering staff, integration work, maintenance, security controls and evaluation. The hardware bill can quickly change the economics of a supposedly inexpensive model.

Kimi K3 makes the point vividly. The model has approximately 2.8 trillion total parameters, including about 104 billion active parameters. Its native MXFP4 checkpoint is approximately 1.56 TB across 96 shards, and Mozilla lists a serving configuration requiring 64 or more accelerators.

So can a typical consumer run Kimi K3 locally at that full configuration? Not realistically. The model’s downloadable weights create deployment flexibility, but “downloadable” is not the same as “fits on a gaming PC.”

Efficient architectures can still reduce the amount of computation activated for each request. Mixture-of-experts designs, for example, route a request through selected parts of a larger model rather than activating everything at once. That can improve serving economics, but it does not erase memory, hardware and operational requirements.

Usage is shifting faster than revenue

Open-weight models already have a strong adoption signal. In OpenRouter’s August 2026 token-volume ranking, eight of the ten highest-volume models had open weights, and seven of those eight were built in China.

Usage and revenue are different markets, however. Older data covering May through September 2025 assigned 96% of model-layer revenue to closed providers. The two figures describe different periods and metrics, so high token volume in the August 2026 snapshot does not mean open models generated most AI revenue.

The adoption pattern still matters. It suggests that developers are willing to trade some frontier performance for lower costs, deployment choice or access to models they can run beyond a single provider’s product. That trade is especially powerful when “good enough” performance is sufficient for the job.

The strategic issue is concentration

The open-model ecosystem is becoming more dependent on Chinese labs even as the strongest closed frontier systems remain concentrated among U.S. companies. Moonshot AI develops Kimi K3; Z.ai / Zhipu AI develops the GLM family; DeepSeek leads the cited OpenRouter request-volume milestone; and Alibaba develops the Qwen family.

That concentration creates a strategic tension. Open weights can reduce dependence on a single hosted provider, but relying heavily on one national ecosystem can introduce a different kind of dependency around model updates, tooling, deployment expertise and supply chains.

For now, the market is not choosing one category everywhere. The evidence points to a split: open-weight models are closing the measured capability gap and winning substantial usage, while closed systems continue to charge for performance in demanding workloads and for the operational services surrounding the model.