Microsoft Research recently open sourced VibeVoice, a 7 billion-parameter speech AI model. It can transcribe up to 60 minutes of audio files in one processing, and simultaneously complete the three tasks of speaker identification, timestamp annotation and text transcription. It supports more than 50 languages, is completely free, and can be executed locally. For developers who are accustomed to the “segment processing” mode of traditional ASR models such as Whisper, VibeVoice’s “one-time full-text processing” solution provides a different approach.
Microsoft VibeVoice-ASR: Transcribe 60 minutes of audio files in one go
When traditional ASR models such as OpenAI Whisper process long audio files, they must cut the audio files into short segments of about 30 seconds and then identify them one by one, and finally splice the results. The fatal flaw of this approach is that the context between each paragraph will be lost, the identity of the speaker may be confused between paragraphs, and the semantic coherence will be interrupted. VibeVoice-ASR operates at an ultra-low frame rate of 7.5 Hz through a continuous voice tokenizer, swallowing 60 minutes of continuous audio files at a time within the context window of 64K tokens, maintaining the global context from beginning to end.
This means that when the model is processing a 40-minute meeting recording, the speech content at minute 35 can refer to the context at minute 5 to determine the reference relationship, and the previous text will not be missed due to slicing boundaries. For long conversation scenarios such as multi-person meetings, podcasts, interviews, etc., the retention of global context directly affects the recognition accuracy.
Three-in-one output: who said what and when
The output format of VibeVoice-ASR is different from the traditional ASR model which only gives plain text. It integrates the three tasks of automatic speech recognition (ASR), speaker separation (Speaker Diarization) and timestamping (Timestamping) into a single end-to-end generation task, and outputs structured transcription results, including three dimensions: “Who (Speaker), When (Timestamps), and What (Content)”.
In practical applications, users do not need to install additional independent speaker separation tools such as pyannote, nor do they need to manually align timestamps after ASR output. VibeVoice-ASR accomplishes these three things simultaneously in one inference, eliminating the need for post-processing. In addition, the model supports customized keywords (hotwords). Users can provide specific names, technical terms or background information to guide the identification process, significantly improving the accuracy of terminology identification in professional fields. For example, common drug names, clinical abbreviations in medical conference recordings, or professional terms used in legal trials can be pre-entered into the model through the hotwords mechanism to reduce the chance of proper nouns being misidentified as near-synonymous words.
Model family: not just identification
VibeVoice is a family of models, not just ASR recognition models. In addition to the ASR recognition model (7B), it also includes a TTS text-to-speech model (1.5B) and a real-time streaming TTS model (Realtime-0.5B). The TTS model can synthesize up to 90 minutes of multi-person conversation speech in one generation, supporting up to 4 speakers. The paper has been accepted by ICLR 2026 Oral. The live streaming version supports streaming text input with approximately 300 milliseconds of latency.
However, the VibeVoice-TTS code was removed from GitHub by Microsoft in September 2025. The official statement pointed out that after the model was released, it was discovered that someone was using it for purposes inconsistent with the original intention of the research. Based on the principle of responsible AI, Microsoft chose to remove the TTS code. The ASR and Realtime-0.5B models continue to remain open source.
GitHub 50,000 stars: community adoption is astonishing
As of August 2025, when it was first open sourced, VibeVoice has accumulated more than 50,000 stars and more than 5,600 forks on GitHub. In March 2026, VibeVoice-ASR was incorporated into the Hugging Face Transformers library, and developers can directly transformers Library calls, no separate installation required anymore. Microsoft also provides vLLM inference support and fine-tuned code to lower the deployment threshold.
In terms of community, Marktechpost has released a complete Google Colab practical tutorial, covering the complete process from environment construction, speaker-aware transcription, keyword-guided identification, batch audio file processing, to real-time TTS synthesis and end-to-end speech-to-speech pipeline. Already developers have created a macOS and Windows voice input application called Vibing based on VibeVoice. Microsoft also provides the ASR Playground online trial environment, allowing users to upload audio files to test the effect without deployment.
VibeVoice GitHub repository
Differences from Whisper: Context is key
The core difference between VibeVoice-ASR and OpenAI Whisper lies in the context processing strategy. Whisper cuts the audio file into short segments for processing, recognizes each segment independently, and then splices the result. VibeVoice-ASR processes the entire audio file in a single inference, preserving global context. In a multi-speaker scenario, Whisper’s slicing method can easily lead to confusing speaker identities across segments, and an additional independent speaker separation model is required to remedy this. VibeVoice-ASR has speaker recognition built into the model itself, without the need for plug-ins.
But VibeVoice-ASR is not without its challenges. The model volume of 7 billion parameters means that local deployment requires sufficient GPU memory. Community testing shows that using the vLLM inference framework can improve the inference speed, but the hardware threshold is still higher than various lightweight versions of Whisper. In addition, ModelScope’s recently released MOSS-Transcribe-Diarize 0.9B model claims to surpass VibeVoice-ASR on benchmarks such as AISHELL-4 and Alimeeting with less than one-eighth the number of parameters of VibeVoice, showing that competition in the field of long-sound file transcription is accelerating. VibeVoice-ASR still maintains its advantages in multi-language support and custom keywords, but whether the parameter efficiency is enough to maintain its lead will be the focus of follow-up observation.
Conclusion
VibeVoice-ASR represents the transformation of speech recognition technology from “segmented splicing” to “global integration”. With a 60-minute processing time, three-in-one structured output, support for more than 50 languages, plus MIT open source license and Hugging Face Transformers integration, it has accumulated 50,000 GitHub stars in one year. With the influx of competitors and the gradual lowering of hardware thresholds, the threshold for long-sound speech recognition is rapidly being lowered. For users who need to process meeting recordings, podcast transcriptions, and interview editing, there are an increasing number of free and locally executed options.
Source: KOCPC Chinese