NVIDIA Research recently open-sourced LongLive 2.0, a long video generation infrastructure built around NVFP4 (4-bit floating point) quantization, spanning the full pipeline from training to inference. According to official data, the 5B parameter model achieves 45.7 FPS inference speed on GB200, generating 64-second videos end-to-end in just 36.3 seconds—over 3x faster than the BF16 baseline’s 112.9 seconds. On the memory front, NVFP4 KV Cache reduces peak VRAM to 19.4GB, opening up long video generation capabilities for consumer-grade graphics cards.
NVIDIA Open-Sources LongLoRA 2.0: NVFP4 Quantization Cuts VRAM Usage in Half, Doubles Speed
This project originated from NVIDIA’s LongLive series, with the first generation accepted by ICLR 2026 in January 2026, focusing on real-time interactive long-form video generation. LongLive 2.0 shifts its focus from “algorithms” to “infrastructure,” attempting to address the speed and memory bottlenecks faced by long-form video generation during actual deployment.
NVFP4 Quantization: Reducing from 20GB to Under 10GB
LongLive 2.0’s core technical breakthrough lies in NVFP4 quantization. Traditional video generation models use BF16 (16-bit floating point) computation, where a 5B parameter model requires approximately 20GB of VRAM, and with KV Cache and increasing video sequence length, it quickly exceeds the capacity of a single consumer GPU. LongLive 2.0 compresses the model to 4-bit precision through NVFP4’s W4A4 (Weight 4-bit, Activation 4-bit) inference pipeline, reducing VRAM usage to under 10GB.
Quality loss after quantization is well controlled. According to the officially published VBench evaluation, LongLive 2.0 scores 85.06 on short video benchmarks, with the gap from the BF16 baseline within acceptable range. Additionally, the team further optimized the NVFP4 inference path with improvements including fused Triton RoPE/adaLN kernels, reduced KV Cache synchronization overhead, and in-place quantized KV Cache updates, resulting in an overall throughput improvement of 18.6%.
Sequence Parallel Training: Preventing Out-of-Memory Errors When Fine-Tuning Long Videos
Autoregressive (AR) training of long videos has always been a memory hog. Models need to process sequences of “clean historical frames” and “noisy target frames” simultaneously. As video length increases, GPU memory, VAE latent encoding, and DiT matrix multiplication costs all surge. LongLive 2.0 introduces a Balanced Sequence Parallel (Balanced SP) mechanism that splits the sequence along the time axis across multiple GPUs, with each card simultaneously holding clean and noisy latent tensors from the same time chunk.
This design offers two advantages. First, it naturally aligns with teacher-forcing attention masks, allowing each GPU to compute valid loss gradients. Second, VAE encoding can also be sharded to avoid single-GPU bottlenecks. With NVFP4 precision, the 64-second AR training achieves a 2.1× speedup compared to the BF16 baseline, with peak memory usage reduced to 0.69× that of BF16.
Multi-camera continuous generation and asynchronous decoding
LongLive 2.0 enables automatic generation of long videos with smooth, seamless multi-camera transitions. Using the Multi-shot Attention Sink mechanism, the model maintains contextual coherence during camera switches, eliminating the frame discontinuities and style shifts commonly seen in traditional methods during shot transitions.
Another optimization on the inference side is Async Decoding. In the traditional pipeline, VAE decoding blocks the generation of the next frame. LongLive 2.0 places VAE decoding on a separate thread, running in parallel with DiT inference to reduce pipeline idle time. Combined with sequence parallel inference and FP8 PTQ (post-training quantization) support, it can run efficiently from single GPU to multi-GPU configurations.
Difference from competitors: Infrastructure mindset vs. model mindset
Currently, the main players in the open-source video generation field include Tencent’s HunyuanVideo, Zhipu’s CogVideoX, and Alibaba’s Wan series. These projects focus on model architecture innovation and video quality improvement, while LongLive 2.0 is positioned differently—it’s an “infrastructure layer” that addresses the challenge of efficiently training and deploying long video models.
LongLive 2.0 itself performs AR fine-tuning based on the Wan2.2-TI2V-5B model, and on May 30 added support for AR teacher-forcing training and DMD distillation for Wan2.2 I2V (image-to-video). This means developers can apply LongLive 2.0’s NVFP4 pipeline to different base models rather than being tied to a single model architecture.
actual deployment performance
Below is a comparison of LongLive 2.0’s end-to-end generation time across different video lengths:
- 16-second videoBF16 baseline: 26.6 seconds → LongLive 2.0 (2-step): 11.2 seconds
- 32-second videoBF16 baseline: 53.2 seconds → LongLive 2.0 (2-step): 19.2 seconds
- 64-second videoBF16 baseline: 112.9s → LongLive 2.0 (2-step): 36.3s
In burn-in testing on the GB200, NVFP4 inference reaches 45.7 FPS, while the standard 5B model inference achieves 24.8 FPS with a VBench score of 85.06.
Research Team and Paper Background
The research team behind LongLive 2.0 is large, consisting of 15 researchers. The corresponding authors include Song Han and Enze Xie from NVIDIA. Song Han is a professor of electrical engineering at MIT and also serves as a research scientist at NVIDIA. He is renowned for his research in model compression and efficient inference, having proposed important theories such as the Lottery Ticket Hypothesis. Enze Xie is a senior researcher at NVIDIA, leading the development of the Sana series of efficient image generation models.
The paper was published on arXiv in May 2026 (number 2605.18739the first generation LongLive was accepted by ICLR 2026 in January 2026. Looking at the timeline, NVIDIA completed the iteration from 1.0 to 2.0 in less than half a year, which is quite fast.GitHub repositoryAs of July 11, it has accumulated 2,437 stars and 229 forks, with enthusiastic community response.
Practical significance for developers
LongLive 2.0 adopts a fully open-source strategy, making everything from model weights and training code to inference pipelines publicly available. Developers can obtain all the code with a single command:git clone --single-branch --branch main --depth 1 https://github.com/NVlabs/LongLive.gitThe project also provides complete documentation, a demo page, and YouTube demo videos, lowering the technical barrier to entry.
For developers interested in deploying long video generation locally, the memory savings from NVFP4 quantization are the deciding factor. The 5B model drops from 20GB to under 10GB, meaning a single RTX 4090 (24GB VRAM) can load both the model and KV Cache simultaneously for long video generation at 720p resolution. This previously required datacenter-grade GPUs like A100 or H100.
In addition, LongLive 2.0 introduced FP8 inference support (W8A8) on July 8, offering another option for balancing precision and speed. For users with newer graphics cards (such as the RTX 5090), FP8 can provide better compatibility without sacrificing too much quality.
Conclusion
LongLive 2.0 represents NVIDIA’s strategic move in the video generation space. Rather than releasing another video generation model, NVIDIA chose to approach from the infrastructure layer, addressing the memory and speed issues that developers find most frustrating when generating long videos. NVFP4 quantization brings the VRAM requirements of a 5B model down to a range affordable by consumer-grade GPUs, while sequence parallelism training means long video fine-tuning no longer requires dozens of GPUs. For developers, this is a toolchain ready to use out of the box, rather than yet another research model that requires extensive adaptation work.
Source: KOCPC Chinese