A transcript can get every word right and still leave a meeting assistant with a serious problem: it cannot tell who agreed to do the work. NVIDIA’s Nemotron 3 Diarization, released on September 23, 2026, marks when each speaker is active in live or recorded audio.
The 99.2-million-parameter model supports up to eight speakers, including people talking at the same time. Its weights are available to download, and developers can run it alongside an existing automatic speech recognition (ASR) system. Nemotron 3 Diarization supplies speaker activity and timestamps, not the words themselves.
For teams building call analytics, meeting notes, podcast transcripts, or voice-agent memory, the release offers a component they can test now. NVIDIA also reports a first-place result on a new diarization benchmark. Neither the leaderboard position nor the shortest advertised buffer tells a developer how well a complete application will work on its own audio.
Speaker Labels Are Not Speaker Identities
Diarization answers who spoke when by assigning anonymous channels to stretches of audio. Nemotron 3 Diarization orders those channels by each voice’s first appearance: the first new speaker gets the first channel, the next gets the second, and so forth. That ordering helps it maintain consistent labels across successive chunks of a conversation.
The labels do not identify people. If the model marks a segment as speaker_2, an application needs another source, such as meeting metadata or a separate speaker-verification system, to associate that channel with a named participant. Developers should not assume that speaker_1 in one recording is the same person as speaker_1 in another.
According to the model card, the model accepts 16 kHz mono audio and can return timestamped speaker segments or frame-level activity probabilities across eight channels. Two channels can be active at once, so the output can represent overlapping speech without forcing every moment into a single-speaker label.
A speaker-attributed transcript combines that activity with ASR output. In an ordinary, non-overlapping exchange, an application can align recognized words with the speaker active at the corresponding time. An interruption is harder: timestamps can show that two people spoke simultaneously, but may not establish which overlapping voice produced each recognized word. Teams need to check the combined transcript for those cases, not just the diarization output.
Developers Can Run It Locally Through Three Routes
NVIDIA’s downloadable model card provides examples for NeMo Speech, a Transformers integration, and the native NeMo-Speech.cpp runtime. It lists the weights under the OpenMDW 1.1 license and describes the model as available for commercial and non-commercial use. Teams should still review the license terms before deployment. “Open-weight” describes access to the model, not the terms governing every possible use.
For a Python application already using NeMo, the card’s basic pattern loads the checkpoint, switches it to inference mode, and passes in a recording:
from nemo.collections.asr.models import SortformerEncLabelModel
model = SortformerEncLabelModel.from_pretrained(
"nvidia/Nemotron-3-Diarization"
)
model.eval()
segments = model.diarize(audio=["meeting.wav"], batch_size=1)
for segment in segments[0]:
print(segment)
The model card documents the installation prerequisites and additional inputs, including NumPy arrays and file manifests. When passing an array, supply its actual sample rate to diarize; a sample-rate mismatch can invalidate the timestamps and predictions that follow.
For a command-line starting point, the card shows these commands after installing NeMo-Speech.cpp:
nemo-speech diarize meeting.wav
nemo-speech transcribe meeting.wav --diarize --json
The first performs diarization. The second invokes transcription with speaker tagging, closer to the output most applications want. Developers using Transformers can follow the card’s separate offline and streaming examples. In the streaming example, the application passes the returned speaker cache into the next call so earlier speaker context is not discarded at every chunk boundary.
Each route serves a different integration need, and each requires inspection of the results. A returned segment is a prediction about an anonymous voice over a time interval, not a verified statement that a particular person said particular words.
A 0.32-Second Buffer Is Not a 0.32-Second Transcript
One checkpoint supports several input-buffer configurations. NVIDIA lists an offline-style 30.4-second setting and streaming settings of 1.04, 0.64, and 0.32 seconds. The 0.32-second option is its lowest recommended setting. The model card says an 80 ms buffer is technically possible, but does not recommend it as the minimum configuration.
Those figures measure how much audio the system waits to collect before running a chunk. NVIDIA calculates the buffer as the chunk length plus right context, or future audio supplied to help interpret the current chunk. The settings are expressed in 80 ms frames: a three-frame chunk and one frame of right context produce the 0.32-second buffer.
They are not end-to-end response times. Model computation comes afterward, followed by whatever networking, ASR, word-to-speaker alignment, and application processing the product requires. A voice assistant using the shortest buffer should measure the time until a usable, speaker-attributed result reaches its user, not report 0.32 seconds as its full latency.
Shorter buffers also give the model less incoming audio to work with at each step. NVIDIA’s longer settings allow more context and may suit recorded meetings or batch processing better. The 30.4-second configuration does not impose a 30.4-second recording limit: NVIDIA says chunked inference can process longer audio. Developers choosing among settings should test both timing and assignment quality on the same recordings.
The First-Place Benchmark Has Defined Boundaries
NVIDIA reports that Nemotron 3 Diarization placed first in the initial VoiceArena Diarization-Bench results, with a 14.72% diarization error rate, or DER. It lists 19.3% for the next-ranked system. The reported comparison covers 139 English-language conversations totaling about 22 hours, across 12 systems and 17 system configurations.
Frequently Asked Questions
3 questions
1Does NVIDIA Nemotron 3 Diarization Transcribe Words?
No. Nemotron 3 Diarization predicts when anonymous speakers are active; it does not produce the spoken words. To make a speaker-attributed transcript, pair its timestamps with an automatic speech recognition system. Overlapping speech needs particular scrutiny because matching recognized words to the correct simultaneous speaker can be harder than marking both speakers active.
2
Sources
- NVIDIA’s Nemotron 3 Diarizationhuggingface.co
- model cardhuggingface.co
