NVIDIA Nemotron 3 Nano Omni: Long‑Context Multimodal Intelligence
Download Nemotron 3 Nano Omni checkpoints from HuggingFace, set up the C‑RADIOv4‑H vision encoder, Parakeet‑TDT‑0.6B‑v2 audio encoder, and enable Efficient Video Sampling for high‑throughput inference.
Set up the model with the new encoders, enable EVS for video, and test on long documents and audio‑video workloads.
Summary
NVIDIA’s Nemotron 3 Nano Omni, released 28 April 2026, is a new omni‑modal model that extends the Nemotron line to support text, image, video, and audio tasks with long‑context reasoning. The architecture combines a hybrid Mamba‑Transformer‑Mixture‑of‑Experts backbone, a C‑RADIOv4‑H vision encoder with dynamic resolution, and a Parakeet‑TDT‑0.6B‑v2 audio encoder that handles up to 20 minutes of native audio. Video processing uses a Conv3D tubelet embedding that fuses consecutive frames, halving the number of vision tokens, and an Efficient Video Sampling (EVS) module that prunes static tokens to improve throughput. Nemotron 3 Nano Omni achieves 9× higher throughput and 2.9× faster single‑stream reasoning compared to alternatives, and it tops benchmarks such as MMlongbench‑Doc, OCRBenchV2, WorldSense, DailyOmni, and VoiceBench. The model supports documents up to 100+ pages, long audio‑video recordings, and agentic GUI tasks, making it suitable for enterprise document analysis, transcription, and multimodal reasoning.
Checkpoints in BF16, FP8, and NVFP4 formats are available on HuggingFace, and the model’s training recipe includes staged multimodal alignment, context extension, preference optimization, and multimodal reinforcement learning. The release also highlights the model’s efficiency gains on long‑document and video use cases, with a 7.4× higher system efficiency for multi‑document scenarios and 9.2× for video.
The announcement positions Nemotron 3 Nano Omni as a cost‑efficient, high‑throughput solution for complex multimodal workloads, with a focus on preserving fine visual detail and native audio understanding.
Key changes
- Hybrid Mamba‑Transformer‑MoE backbone with 23 Mamba layers, 23 MoE layers, and 6 grouped‑query attention
- C‑RADIOv4‑H vision encoder with dynamic resolution for dense documents
- Parakeet‑TDT‑0.6B‑v2 audio encoder for native audio up to 20 min
- Conv3D tubelet embedding for video, halving vision tokens
- Efficient Video Sampling (EVS) prunes static tokens to improve latency
- 9× higher throughput and 2.9× faster single‑stream speed vs alternatives
- Supports 100+ page documents, long audio‑video, and agentic GUI tasks
- Top accuracy on MMlongbench‑Doc, OCRBenchV2, WorldSense, DailyOmni, VoiceBench