NVIDIA Releases Nemotron 3.5 Super VL, a 120B Open Multimodal Agent Model

When NVIDIA drops a 120‑billion‑parameter multimodal model and calls it “always‑on enterprise agent,” the first thing I ask is: how does this affect the real‑world cost of running an agent stack for a midsize hosting outfit? The Nemotron 3.5 Super VL announcement reads like a sales sheet – 120 B parameters, 12 B active per token, 1 M‑token context, video support – but the devil is in the deployment details, the hardware bill, and the performance claims that actually matter when you’re serving paying customers.

What the model actually is

Nemotron 3.5 Super VL is an open‑weight vision‑language model that merges text, image, document and short‑clip (MP4 up to two minutes) inputs into a single output stream. NVIDIA ships the weights under the OpenMDW‑1.1 license, so you can pull the checkpoint from Hugging Face or their NIM service without a proprietary lock‑in.

The architecture is a hybrid of Mamba‑2, mixture‑of‑experts (MoE) and traditional attention layers. Each MoE layer contains 512 experts, but only 12 B of the total 120 B parameters are active for any given token. A C‑RADIO vision encoder and connector stage feed visual data into the language backbone, and the model can toggle reasoning on or off via the chat template. In short, it’s a single checkpoint that replaces the usual two‑model stack of a text‑only LLM plus a separate vision encoder.

Training pedigree and data scale

NVIDIA says the model was distilled from its Nemotron 3 Ultra predecessor and trained on roughly 26 trillion tokens. After the main pre‑training run they added about 3.8 million samples plus reinforcement‑learning fine‑tuning across dozens of environments, then a further 10 B Indic‑language tokens. Those numbers are impressive on paper, but they also hint at the massive compute budget required to get a model of this size to a usable state – a budget most independent hosting providers simply can’t match.

For context, the Ultra model carries about 4.6 × more total parameters than Super VL, yet Super VL still outperforms it on the Hermes and LangChain benchmark suites (63.44 % vs 56.33 % on Hermes; 87.05 % vs 84.3 % on LangChain). The performance edge comes from the MoE routing and the new decoding tricks NVIDIA calls “Multi‑token prediction” and “LiLiCorr.”

Speed tricks that matter to us

The paper‑thin “up to 5× higher single‑GPU throughput than open peers” claim rests on a stack of speculative decoding techniques. Multi‑token prediction drafts several tokens in one forward pass, while LiLiCorr, a speculative drafter for DGX Spark, promises up to 2.3× faster token generation. NVIDIA also bundles a quantization‑aware distillation called NVFP4, which runs at 4‑bit weight precision (with 16‑bit activations on non‑Blackwell GPUs, 4‑bit activations on Blackwell). In their own testing they logged over 600 output tokens per second on an “Artificial Analysis” agentic task via a Baseten endpoint, and a 5× speed gap over the Inkling Small reference model.

Those numbers look good until you factor in the hardware required. The reference deployments listed are 1 × DGX Spark, 1 × H200, 2 × H100, or 1 × GB200. Even the “edge” Bolt variants need a 2‑bit quantized 24 GB footprint to run on an RTX‑based system. For a hosting provider that typically runs on commodity x86 servers with 64‑GB RAM, the cost of provisioning a DGX‑class box just to host a single agent is prohibitive.

Benchmark scores – take them with a grain of salt

The model card lists a suite of scores from the NVFP4 checkpoint: 89.90 on GPQA‑Diamond, 71.07 on SWE‑bench Verified, 77.5 on BrowseComp, and a low 17.78 on a banking‑focused tau3‑bench. NVIDIA also compares Super VL against its own Ultra model, showing a modest edge on a few benchmarks. However, the article reminds us that “scores are self‑reported” and that harnesses differ across vendors, so a few points swing isn’t a decisive win.

What matters for us is the “agent harness” performance. NVIDIA claims a 63.44 % Hermes score and 87.05 % LangChain score, which are respectable but still leave a large gap to human‑level reasoning. In production, you’ll see those percentages translate into occasional hallucinations or tool‑calling failures – exactly the kind of risk that can break a SaaS contract.

Deployment realities and cost implications

Super VL is available through multiple runtimes: vLLM, SGLang, TensorRT‑LLM on Blackwell/Hopper, and even llama.cpp or Unsloth for local inference. NVIDIA’s “Dynamo recipes” add KV‑aware routing for production, and they claim an 8× increase in users served on a 16‑GPU B200 cluster at 50 tokens / second per user, with first‑token latency dropping 85 %.

Those figures are impressive in a lab, but they assume you can afford a 16‑GPU B200 rig – a machine that costs well into six‑figures. By contrast, most independent providers run on clusters of 8‑core CPUs with a few mid‑range GPUs (e.g., RTX 3080‑ti). To get comparable throughput you’d need to shard the model across multiple nodes, adding networking overhead and complexity. The “single‑DGX Spark” deployment option still requires a DGX‑class box, which is out of reach for most SMBs.

Edge models – a more realistic path?

The Nemotron 3.5 Bolt 74 B‑A9B model is positioned as a “local agent” variant. It uses 2‑bit quantization to fit into 24 GB of memory, making it runnable on a single RTX‑3080‑ti or a DGX Spark with reduced precision. NVIDIA touts it as #1 on PinchBench among open models in its class and #2 on BrowseComp. The Bolt E4B 4 GB variant, arriving October 23, targets voice assistants on Jetson and other embedded platforms.

From a hosting perspective, the Bolt line is the only part of the release that could be realistically offered as a managed service on commodity hardware. You could spin up a fleet of RTX‑based servers, run the Bolt model with 2‑bit quantization, and expose an API that handles text‑plus‑image queries. The trade‑off is reduced model capacity (74 B total vs 120 B) and likely lower performance on complex reasoning tasks, but the cost per node drops dramatically.

Bottom line and actionable advice

Nemotron 3.5 Super VL is a technically impressive multimodal agent, but its hardware demands keep it firmly in the hyperscaler or large‑enterprise arena. For independent hosting providers, the sensible move is to ignore the flagship 120 B MoE checkpoint and focus on the Bolt 74 B‑A9B variant, which fits on affordable GPUs with 2‑bit quantization. Pair it with NVIDIA’s open‑source training recipes if you need domain‑specific fine‑tuning, but budget for the compute and storage required for the full 120 B model.

In practice, start by benchmarking the Bolt model on your existing GPU fleet, measure latency and token‑throughput, and compare that against your current LLM stack. If the latency meets your SLA and the cost per request is lower than running a separate vision encoder plus text model, then you have a win. Otherwise, stick with your proven two‑model pipelines and keep an eye on NVIDIA’s roadmap – the next iteration may finally bring MoE‑scale reasoning to commodity hardware.

In short: the Super VL hype is real, but the real‑world risk‑reward balance still favors the smaller, quantized Bolt models for anyone not sitting on a DGX‑Spark budget.

— Allan Ali, Founder

This article was produced with AI-assisted research and editorial support. Reporting is based on the source material cited below. Sources: MarkTechPost; marktechpost.com; Global1.News (17 October 2026).

By Allan Ali, Global1.News