Open weights, open questions: what the last month of releases actually shows

Mozilla published version 1.1 of its State of Open Source AI report on 15 September, using data current to 1 September, according to Tom's Hardware. The headline finding: the best open model trailed the closed leader on the Artificial Analysis Intelligence Index by three points at 60% of the price, and sat two points behind Claude Fable 5 at 30%. Mozilla's fit on METR task-horizon data puts the open-closed gap at around 4.4 months, in line with Epoch AI's four-month estimate.

That is a compressed gap. It is also measured in a very specific way.

What the four-month figure actually measures

The number rests on METR, a research nonprofit that scores models by the length of task, in human working time, that they complete half the time. By Mozilla's fitted estimate, closed models handle tasks that take human experts 8 to 12 hours. Open models reach that level about four months later. Mozilla computes open capability doubling every 3.9 months against 5.5 months for closed models. The report is built on a Mozilla/SlashData survey of roughly 1,400 developers, plus OpenRouter traffic data and third-party benchmark indices, Tom's Hardware reported. Mozilla is the nonprofit behind Firefox and an advocate for open models: TIME reported on 14 July that Mozilla chief technology officer Raffi Krikorian described the report as partly advocacy.

There is a second caveat buried in the methodology. The four-month gap and the 30% token price figure are measured API to API on hosted endpoints and at list price. That is a comparison of how two categories of model are sold, not only of what they can do.

The report's own hardware chart sharpens the point. The best open model that fits on one server scores 52.6; the best that fits on one GPU scores 40. The drop from the top is 10 and 23 points respectively, a larger gap than the reported four months. Kimi K3's native MXFP4 checkpoint runs about 1.56TB across 96 shards, and Mozilla's serving configuration lists 64 or more accelerators, while vLLM calls for at least eight GB300 GPUs, with multiple nodes for production traffic. The report describes this as open but not runnable by most who hold it. Tom's Hardware put the memory need near 1.5TB in July. One counterexample in the report is Thinking Machines' Inkling-Small, under the Apache 2.0 license, whose NVFP4 version fits one B300 at a 180GB floor.

So: downloadable, yes. Practical for a small team, sometimes not.

The cheap-and-close benchmarks

Mozilla also charted vals.ai's Terminal-Bench 2.1 results, which run every model through the same harness, the software layer that offers a model its tools. On that board, Z.ai's GLM-5.2 scored within a point of Claude Opus 4.7 and about four points behind Opus 4.8, at less than one-fifth the cost per test. On OpenRouter, a marketplace that routes developer traffic to hundreds of models, Mozilla counted eight of the top ten models by August token volume as open weights, seven of them Chinese-built. Closed providers nevertheless took 96% of model-layer revenue on OpenRouter from May to September 2025, according to the Linux Foundation.

Krikorian told Ars Technica in an email that Mozilla sees the decision to pay for closed models as workload-specific rather than organization-specific.

The data stops at 1 September, and the boards have moved since. Artificial Analysis shifted its index to v4.3 with a different evaluation set; the live board has Claude Fable 5.1 at 53 on its highest effort setting with Kimi K3 at 44, numbers not comparable to the v4.1.1 figures Mozilla plotted. vals.ai's Terminal-Bench 2.1 board, updated 11 September, is led by GPT-6 Astra at 87.27% with Fable 5.1 at 85.02%. Mozilla's own chart caption reads: "the gap resets every release cycle."

K3 also carries an allegation detailed in the 8 September NSA/CISA/FBI joint advisory AA26-251A. The claim, which Mozilla's report states as "asserted, and unshown," is that Moonshot extracted Claude Fable 5 data to train K3 through distillation, the practice of training one model on another model's outputs. On 17 July, Artificial Analysis had K3 at 57 versus Fable 5's 60; on 1 September, Mozilla had it two points back.

Open weights are not Open Source, and the distinction is doing work

The Open Source Initiative used the moment to draw a line. In a 19 September post, OSI argued that open-weight models allow you to exercise some of the four software freedoms, the freedom to use, study, modify and share without asking permission from the rights holder, but not all, and only to a certain degree. A model consists of three components: training data, weights and parameters, and code for data preparation and training. Open weights release only the middle one.

With open-weights, you can choose to run a model locally, paying only for the electricity and hardware, or you can work with a third-party provider to host it. You can also fine-tune the model with your own data.

OSI's example of the alternative is Ai2's Olmo, released as Open Source AI, which let researchers inject new information into the training data and watch how the model memorized or forgot it. Mozilla's report counts 16 notable open releases, but Tom's Hardware notes that none delivers the data recipe required by the Open Source Initiative's definition. OSI puts the demand-side value of the Open Source ecosystem at $8.8 trillion.

That framing matters for policy, but it does not settle whether open weights are useful. They plainly are, in narrow jobs.

Small models, specific jobs

Superwhisper released S1-mini, a 0.6B-parameter text normalizer for speech-to-text output, fine-tuned from Qwen/Qwen3-0.6B and licensed Apache 2.0 with a naming clause. It takes a raw ASR transcript and rewrites it as clean written text: fillers removed, false starts resolved, punctuation applied, spoken numbers and dates rendered in written form. On a held-out set of 7,519 English cases it reaches 94.8% token accuracy, and the quantized build is a 462 MiB file that runs on a laptop CPU. The model card is unusually blunt about scope: it is not a chat model, will not follow general instructions, covers English only, and expects a control line at the top of the input. Its parameter count is also a small lesson in reading model cards: the Hub sidebar reports 0.8B, but the card explains that lm_head.weight is stored as a materialized copy of the input embedding, so the 155.6M-parameter embedding is counted twice, 751.6M tensor elements against 596.0M unique parameters.

A second example comes from a practitioner write-up, not a vendor. James Cruce described on 22 September replacing a deterministic router in his own agent system with Laya, an open-weight typed-decision model from convaiinnovations with about 421 million parameters, a ModernBERT-large encoder and a 1,024-token limit. It returns typed answers, a choice among named options, a score on an ordered scale, or a probability for a yes-or-no statement, rather than prose to parse. He chose it over TypeSafe's hosted Jev, which lists $0.042 per million input tokens with no output-token charge and handles choices with up to 255 options, because he wanted routine decisions running locally on a Mac Studio with an M3 Ultra and 256GB of unified memory. He is explicit that this is one deployment: he did not test whether Laya beats Jev in general, and he treats the model as a recommender inside a program, never as a source of permission or truth. The model card warns it is overconfident and needs domain-specific calibration.

These are the cases where open weights stop being a geopolitical argument and start being an engineering choice. A 462 MiB normalizer and a 421M decision model are not frontier systems. They are also the kind of thing a team can pin, run offline and audit without a procurement call.

The copyright question stays unresolved

The most consequential open-weight research of the past year concerns what is inside the weights. A paper by A. Feder Cooper, Mark A. Lemley and seven co-authors, first submitted on 18 May 2025 and revised through 20 July 2026, measures memorization of books across 200 titles and 14 open-weight LLMs, through more than 3000 experiments. The abstract states that most LLMs do not memorize most books, either in whole or in part, but that there are notable exceptions. Llama 3.1 70B entirely memorizes some books, including Harry Potter and the Sorcerer's Stone; the authors write that one can deterministically extract the whole book almost verbatim using the book's first few words as an initial prompt.

The paper's framing is deliberately uncomfortable for both sides of the copyright fights. It states that plaintiffs and defendants make sweeping, opposing claims that dramatically oversimplify the relationship between memorization and copyright, and that its results have significant implications for copyright cases, though not ones that unambiguously favor either side. The paper was accepted at COLM 2026.

Put the two strands together and the month's picture is less tidy than either camp wants. Open weights are close enough to frontier on aggregate benchmarks to reprice a lot of work, and drastically cheaper on some of them. They are frequently too large to serve without a rack. They are not Open Source by the definition a large part of the community uses. They can contain verbatim text from books nobody licensed. And a 0.6B model that cleans up dictation still does exactly what it says.

Mozilla's own caption may be the most honest line in the report: the gap resets every release cycle.