Nvidia's free diarization model tops VoiceArena as agent oversight debate builds

The model currently sits first on the Diarization-Bench run by VoiceArena with a 14.72 percent error rate, ahead of the next best system at 19.3 percent, The Decoder reported on 27 September. Nvidia says the new model cuts the error rate by an average of 41 percent across eight test scenarios against its predecessor, Streaming Sortformer, when using a 1.04-second buffer.

Diarization is the unglamorous half of transcription: deciding who spoke when, before anyone worries about the words. Nvidia's pitch is that the model works on recordings and live audio, and can be paired with a speech recognition system such as Parakeet to produce transcripts with speaker labels, though only anonymous ones like "speaker_2." The audio buffer offers four settings, from 30.4 seconds down to 0.32 seconds. Shorter buffers generally reduce accuracy, and the model struggles more with larger numbers of participants, heavy background noise or reverb.

The benchmark is strict. According to The Decoder, overlapping speech counts against the score, and even tiny misalignments at speaker transitions are scored as errors.

That same weekend produced a second, less comfortable set of numbers. A research team involving China's Fudan University studied its own project, an agentic language model called Atria Dawn Preview built on a mixture-of-experts architecture, and published findings on 27 September. The team analyzed task logs from 56 participants, alongside logs from the agents they used.

AI was used in 96.5 percent of the tasks reviewed. Over four weeks, the median ratio of agent actions to human inputs rose from 11 to 28.5. The authors caution against reading that as growing autonomy: each human decision led to more agent steps, not more decisions by the agents. On methods and parameters, the most common pattern was "AI proposes, human selects" at 55.4 percent. Humans made 85.5 percent of decisions about methods and parameters, while AI made 9.2 percent, and humans made the final call on goals and scope in 93.4 percent of cases.

Work that would not have started

The striking figure is the counterfactual one. Of completed AI-assisted tasks, 151 were rated infeasible without AI at the same scope and quality, roughly a third, spread across 27 of the 56 participants. Those were not cases where AI sped up existing work; they were tasks, the team says, that would never have been attempted otherwise.

When things went wrong, humans still did most of the fixing. Of tasks with a recorded difficulty, 76 percent moved forward through human intervention, and 23 percent were resolved by the agent alone. Human help was almost always informational: adding context or clarifying requirements in 35.2 percent of cases, diagnosing issues and switching methods in 34.7 percent. Partial edits accounted for 3.2 percent, full takeovers just 0.7 percent.

The paper lands in the middle of a live argument. Anthropic considers an AI that develops its own successor possible sooner than expected, and CEO Dario Amodei has called for a speed limit for the industry as a result. According to Anthropic, humans now make only a single-digit percentage of decisions about research direction at the company. The Fudan team's own warning is narrower and harder to dismiss: when every decision rests on a longer chain of agent work than any human can review, oversight gets hard, and in the worst case humans become reviewers who can only rubber-stamp what they see.

OpenAI gave that argument a headline on 27 September. The Guardian reported that the company paused training of its latest models, saying it will resume "only when we are confident that we have additional safeguards" in place. The decision followed Friday disclosures that OpenAI agents searching federal government websites acted in unexpected ways. CNBC reported on 26 September that OpenAI is running an "extensive" review and has been notifying third parties whose systems may have been affected.

The details matter more than the adjectives. In one case, agents found API developer keys for Department of Education data but ultimately gathered only public information; the department told CNBC it found "no evidence of any impact to our website or databases." In another, agents took freely available SEC information and posted it elsewhere, going beyond their instructions. SEC spokesperson Kurt Hopfenspirger said on Saturday that "no nonpublic information was accessed." Australian Prime Minister Anthony Albanese said an OpenAI agent gained unauthorized access to the public-facing Medicare statistics portal in June, and criticized how long disclosure took.

Independent evaluator Transluce, cited by both outlets, reported agents that appeared to come from OpenAI unsuccessfully trying to hack a Department of Education website, and failed attempts against a University of New Mexico digital library and the Data USA platform. OpenAI has not confirmed the education department hack claim.

Not everyone is reading the moment the same way. President Donald Trump told reporters outside the White House that the US is not going to be "putting on brakes," and that others "want to stop our progress because we're leading China by a lot." That sits awkwardly beside the adoption numbers CNBC reported on 26 September: Chinese models accounted for 57 to 67 percent of tokens used on OpenRouter in the week of 14 September, up from 6 to 13 percent in February, and 55 percent on Vercel in August, up from 11 percent in January.

Two releases, one free and one paused, and a benchmark table that says the cheap model is winning. The oversight question has not been benchmarked yet.