AI Wiki

Nemotron 3.5 Lightning

AI ModelsLarge Language ModelsMixture of ExpertsNVIDIAOpen Source AI
15 min read
Updated
Suggest editHistoryTalk19 citationsFact-checked
RawGraph
FieldValue
DeveloperNVIDIA
ReleasedAugust 11, 2026
TypeOpen-weights large language model (mixture-of-experts)
Parameters30B total, 3B active (31.6B / 3.6B counting embeddings)
ArchitectureHybrid Mamba-2 + MoE + attention, with multi-token prediction
Context windowUp to 1 million tokens
LicenseOpenMDW-1.1
PredecessorNemotron 3 Nano (30B-A3B)
Weightshuggingface.co/nvidia (BF16, NVFP4, Base, DSpark, DFlash)

NVIDIA Nemotron 3.5 Lightning is an open-weights 30 billion parameter mixture-of-experts language model with 3 billion active parameters per token, released by NVIDIA on August 11, 2026 under the permissive OpenMDW-1.1 license. [1][2][3] It introduces the 3.5 designation and the "Lightning" tier name to NVIDIA's flagship open-model line; NVIDIA positions it not as a frontier reasoner but as the high-volume execution layer of long-running AI agents, handling the tool calls, result validation, and subagent delegation that dominate an always-on agent's token budget while larger models such as Nemotron 3 Ultra handle planning. [1] NVIDIA says the model delivers up to 4 times the output speed of similar-sized models, and the independent benchmarking firm Artificial Analysis measured median output speeds of nearly 670 tokens per second on a pre-release DeepInfra endpoint, alongside an Intelligence Index score of 24 that matches OpenAI's much larger gpt-oss-120b. [1][2][12]

The release came paired with NVIDIA NeMo Switchyard, an open-source model-routing library that sends each step of an agent workflow to the most capable and efficient model available, with Lightning intended as the low-cost target that routine work routes down to. [1][10] Lightning also marks a licensing shift for NVIDIA's small models: where Nemotron 3 Nano and Super shipped under the NVIDIA Nemotron Open Model License, Lightning follows Nemotron 3 Ultra in adopting OpenMDW-1.1, a short permissive license with no use restrictions. [3][8]

What is Nemotron 3.5 Lightning?

Nemotron 3.5 Lightning is a text-only reasoning and chat model built for what NVIDIA calls the execution layer of agentic AI. The company's framing is that long-running agents spend most of their time on high-volume execution steps ("requests like git pull, validate tool outputs, format results") and that sending every such step to a frontier reasoning model adds unnecessary cost and latency. [1] Lightning is meant to absorb that traffic. NVIDIA describes it as designed for agent harnesses "like OpenClaw and Hermes Agent," both supported by the NVIDIA NemoClaw open-source security and management stack for always-on agents, and says the model received harness-optimized training so that agents make more accurate calls at lower latency. [1]

Despite the 3.5 version number, NVIDIA presents Lightning as "the smallest member of the Nemotron 3 model family," effectively the successor to the December 2025 Nemotron 3 Nano, which used the same 30B-A3B configuration. [1][12][17] Artificial Analysis, which tested the model at launch, called it "the first Nemotron 3.5 model" and reported an Intelligence Index gain of 9 points over Nemotron 3 Nano (24 versus 15), putting it just behind Nemotron 3 Super (26), a model roughly four times its size. [12]

The model supports a context window of up to 1 million tokens, with 256K used as the default for single-GPU deployments. Reasoning can be toggled on or off through the chat template (enable_thinking), with a runtime thinking budget also exposed. Supported natural languages are English, Spanish, French, German, Italian, and Japanese; the training corpus spans 20 spoken languages and 43 programming languages, and post-training additionally covered Chinese. [3]

Architecture

Lightning keeps the hybrid design NVIDIA has used across the Nemotron 3 family: interleaved Mamba-2 state-space layers and mixture-of-experts feedforward layers, with a small number of attention layers. [3][12] The MoE routing means only about 3 billion of the 30 billion parameters run per token, which gives the capacity of a larger dense model at the compute cost of a small one. [1] Counting embeddings, the model has 31.6 billion total and 3.6 billion active parameters, the same convention gap seen with Nemotron 3 Nano. [12]

The network also carries built-in multi-token prediction (MTP) layers, which predict several future tokens per position. MTP was baked in during pre-training, as with Nemotron 3 Super and Ultra, and a dedicated MTP-boosting phase after training further improved its accuracy. [1][3]

Speculative decoding: MTP, DSpark, and DFlash

A distinctive part of the release is that Lightning ships with three speculative decoding options rather than one. [1][3]

MethodWhat it isRecommended for
MTPBuilt-in layers that predict several future tokens at each positionMedium to high concurrency serving; optimal draft length shrinks as concurrency rises
DSparkExternal semi-autoregressive drafter that proposes a whole block of candidate tokens in one forward pass from a parallel backboneDGX Spark and low-concurrency data center deployments (NVIDIA's current default recommendation)
DFlashExternal drafter using a lightweight block-diffusion model to generate an entire draft block in one forward passWorkload-dependent; released for users to measure against the others

DSpark and DFlash are published as separate checkpoints on Hugging Face alongside the main weights. [5][6] For maximum-throughput batch serving on a single H100, NVIDIA's own recipes recommend no speculative decoding at all. [4]

Released checkpoints

RepositoryContents
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16Full-precision reference weights, intended for customization, post-training, and building quantized variants [3]
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP44-bit deployment checkpoint, the recommended path for production inference [4]
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16Pre-trained base model before supervised fine-tuning and RL [7]
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSparkDSpark draft model for speculative decoding [5]
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlashDFlash draft model for speculative decoding [6]

The NVFP4 checkpoint was produced by post-training quantization with NVIDIA's Model Optimizer, using a "Four Over Six" NVFP4 recipe (a variant of static MSE calibration) with W4A16 on the routed and shared experts and FP8 dynamic scales on the Mamba projections and KV cache, calibrated on 1,000 samples of 32K tokens from the Nemotron Ultra validation set. [4] The same file serves through native FP4 kernels on Blackwell GPUs and through W4A16 kernels on Hopper and Ampere hardware, which lack FP4 tensor cores. [1][4] Artificial Analysis measured the NVFP4 variant at the same Intelligence Index score of 24 as the BF16 weights, calling the quantization near-lossless. [12]

How was it trained?

According to the Hugging Face model card, Lightning was built between December 2025 and May 2026 in four stages. [3] Pre-training ran for more than 20 trillion tokens using an NVFP4 recipe on the Nemotron 3 data foundation (Nemotron-CC-v2/v2.1 web data, Nemotron code and math corpora, and synthetic specialized-domain sets), with a pre-training data cutoff of September 2025. A continued pre-training phase then aligned the MTP layers with the base model. Supervised fine-tuning followed on synthetic code, math, science, tool calling, instruction following, and structured-output data. Finally, the model underwent multi-environment reinforcement learning with GRPO across math, code, science, instruction-following, multi-step tool-use, multi-turn conversation, and structured-output environments, using an asynchronous RL architecture that decouples training from inference and uses MTP to speed up rollout generation. Post-training data extends to May 2026. [3]

Pre-training used Megatron-LM; reinforcement learning used NVIDIA's open NeMo RL and NeMo Gym libraries, and the evaluation recipes for reproducing the reported benchmark scores are published in the NeMo Gym repository. [3][19] Keeping to the pattern of earlier Nemotron launches, NVIDIA released the training data and recipes along with the weights, including a new open agentic RL dataset, Nemotron-RL-Agentic-Terminal-Pivot-v1 (CC BY 4.0), used to train some of the model's coding-agent behavior. [1][9] The model card also discloses that post-training data was distilled from teacher models including GPT-OSS-120B, and that NVIDIA filtered out teacher-generated reasoning traces that "implicitly align with specific political entities or promote nationalistic narratives." [3]

How does it perform on benchmarks?

NVIDIA's headline claims are about speed per unit of accuracy rather than raw scores. The company says Lightning "delivers leading accuracy at the highest output speed in its class, winning the accuracy-versus-speed Pareto frontier on the Artificial Analysis Intelligence Index," and that on PinchBench, an agent benchmark, the model reaches 86 percent accuracy while completing 10,000 tasks 30 percent faster than Qwen3.6 35B at similar accuracy. [1]

NVIDIA's own accuracy numbers, measured under a consistent NeMo Gym / NeMo Evaluator harness, place Lightning below the best similar-sized open models on most static benchmarks while well above its predecessor: [3]

BenchmarkLightning (BF16)Qwen3.6 35B A3BGemma 4 26B A4BNemotron 3 NanoNemotron 3 SuperGPT-OSS 20B
MMLU-Pro81.9485.6385.2078.4683.8976.40
GPQA Diamond (no tools)75.4483.4079.6174.0578.6071.46
HLE (text-only, no tools)11.7219.5617.4210.8920.3013.76
SciCode32.6035.3340.2830.0835.1138.63
SWE-bench Verified51.5670.1257.4034.0863.0852.44
Terminal-Bench 2.124.5844.3837.228.2939.6115.17
PinchBench85.3788.0774.7066.1180.3657.20
BrowseComp36.9748.7426.3013.7422.77not reported
GDPval-AA-V2 (Elo)8321015807473746not reported
IFBench (loose)71.8863.7177.2572.1771.9268.50

Artificial Analysis's independent launch assessment on August 11, 2026 filled in the efficiency side of the argument. On its Intelligence Index (nine combined evaluations), Lightning scored 24, a 9-point jump over Nemotron 3 Nano, level with gpt-oss-120b, and just behind Nemotron 3 Super at 26, but behind the strongest small open models, Qwen3.6 35B A3B at 32 and Meta's Muse Glimmer (high) at 35. [12] The firm's time-per-task measurements are where Lightning stood out: roughly 0.5 minutes per Intelligence Index task on the pre-release DeepInfra endpoint, against about 3.5 minutes for Qwen3.6 35B A3B, 3.4 for gpt-oss-120b, and 5.8 for Gemma 4 31B. [12] It also noted that proprietary models still hold the overall time-efficiency frontier, with Gemini 3.5 Flash-Lite scoring 37 at a similar time per task. [12] The biggest gains over Nano came on agentic evaluations: a GDPval-AA v2 Elo of 824 in Artificial Analysis's measurement (up 334 points, moving past gpt-oss-120b and Nemotron 3 Super) and Terminal-Bench 2.1 at 24 percent versus Nano's 7 percent. [12] German outlet The Decoder summarized the release as a model that "prioritizes speed over maximum intelligence." [13]

Deployment and hardware

The BF16 weights run on a single H100 80GB or A100 80GB; the NVFP4 checkpoint targets a single DGX Spark (GB10) or H100, and also runs on GB200 and the GeForce RTX 5090. [3][4] NVIDIA frames the model as sized for local AI, citing Jetson, RTX 5090, and DGX Spark systems, and worked with EXO Labs to characterize DGX Spark performance; NVIDIA says the model sits on the Pareto frontier for small open models on EXO Labs' local.ai leaderboard. [1]

Deployment recipes cover vLLM, SGLang, and TensorRT-LLM; the vLLM configuration uses the nemotron_v3 reasoning parser together with the qwen3_coder tool-call parser. [3][4] The model is also available through the standard local-tooling set, including Ollama (as nemotron-3.5-lightning), llama.cpp, LM Studio, and Unsloth, and hosted on build.nvidia.com and OpenRouter. [1][15][16] At launch, serverless inference was offered by providers including DeepInfra, Fireworks AI, FriendliAI, CoreWeave, GMI Cloud, Nebius, and Crusoe, with Baseten, BlackBox AI, Modal, and Together AI also listed in NVIDIA's partner ecosystem. [1][12]

NeMo Switchyard and model routing

Lightning launched together with NVIDIA NeMo Switchyard, an open-source (Apache 2.0) model-routing library published in the NVIDIA-NeMo GitHub organization. [10][11] Switchyard evaluates each request in an agent workflow and sends it to the model best suited to the task's requirements, cost, and latency constraints, so that "plans route up to the frontier, execution routes down to Lightning." [1] The library is provider-agnostic (its SDK, switchyard-libsy, maps semantic model names to provider endpoints), ships a reference server that accepts OpenAI, Anthropic, and Responses API requests, and offers both tuning-free routers (an LLM classifier, a stage router that tracks a coding agent's phase, and an escalation router that starts cheap and moves up on sustained difficulty) and tunable routers, including a prefill router trained on the model's residual stream. [10]

NVIDIA published two early results for the system-of-models approach. LangChain benchmarked Switchyard on its internal deep agents evaluation suite of 145 multi-turn agentic tasks: routing between Nemotron 3.5 Lightning and Claude Opus 4.8 with the escalation router cut costs 74 percent against a frontier-only baseline, sending just 7 percent of calls to the frontier model, at a measured accuracy tradeoff of about 6 points. [10] Cognition implemented the staged-routing method in Devin Desktop and deployed it to NVIDIA internal users; on its FrontierCode Main coding benchmark, routing between Opus 5 and Kimi K2.7 reached 50.6 percent at a mean cost of $3.11 per task, within 2.8 percentage points of Opus 5 accuracy at roughly 28 percent lower mean cost. [10] Other named Switchyard collaborations include Nous Research's Hermes Agent, Ramp, LiteLLM, Kong, Boomi, Cadence, and Siemens. [10]

Licensing and openness

Lightning is released under the OpenMDW License Agreement, version 1.1, with the license text shipped in the Hugging Face repositories. [3][8] OpenMDW-1.1 grants permission "to deal in the Model Materials without restriction" across copyright, patent, database, and trade secret rights, requires only license and notice retention on redistribution, imposes no restrictions on model outputs, and terminates for licensees who bring patent or copyright suits over the materials. [8] Artificial Analysis described it as "open for commercial use without material restrictions." [12]

This is a more permissive arrangement than the NVIDIA Nemotron Open Model License that governed Nemotron 3 Nano and Super, which carries use restrictions and a termination right for NVIDIA. Nemotron 3 Ultra had already moved to OpenMDW-1.1 at its June 2026 release, and Lightning continues that direction for the family's small models. [3][18] As with earlier Nemotron launches, the release spans weights, training data, and recipes: the pre-training and post-training dataset collections, the Terminal Pivot RL dataset, fine-tuning paths through NeMo Automodel and NeMo Megatron Bridge, and RL tooling through NeMo RL and NeMo Gym. [1][9]

Ecosystem and reception

NVIDIA announced the model with an unusually long partner list spanning post-training services (including Thinking Machines Lab, Prime Intellect, Deep Cogito, and Applied Compute), harnesses and agent frameworks (including Cline, Factory AI, OpenCode, OpenHands, OpenClaw, and Hermes Agent), cloud platforms (Amazon SageMaker JumpStart, Google Cloud Gemini Enterprise Agent Platform, Microsoft Foundry, OCI Enterprise AI), and systems integrators (Accenture, Tata Consultancy Services, Tech Mahindra, Wipro). [1] Artificial Analysis noted that NVIDIA had worked with partners including CodeRabbit and Harvey to post-train Lightning for domain-specific work. [12]

Early coverage read the release through the routing lens. MarkTechPost covered the launch as a two-part story, an execution model plus a router. [14] The Decoder emphasized that the model trades peak intelligence for throughput, calling Artificial Analysis's measured output speed of nearly 670 tokens per second the highest among the models compared at launch. [13] As of August 12, 2026, the day after release, the Ollama package showed about 12,500 pulls and the two main Hugging Face checkpoints a combined 35,000 downloads. [15][3][4]

ELI5

Think of a big AI agent as an office. A few hard problems need the expensive senior expert, but most of the day is routine paperwork: run this command, check that result, file this output. Nemotron 3.5 Lightning is a small, very fast worker that NVIDIA built (and gave away for free, including the training materials) to do exactly that routine work. It is not the smartest model available, but it answers several times faster than models its size, so an agent that runs all day gets much more done per dollar. NVIDIA also released a "dispatcher" called NeMo Switchyard that looks at each task and decides whether to hand it to the fast cheap worker or send it up to an expensive expert model. In tests by LangChain, letting the dispatcher choose cut the bill by about three quarters while giving up only a little accuracy.

See also

References

  1. ^Chris Alexiuk and Chintan Patel. "NVIDIA Nemotron 3.5 Lightning Delivers Fast, Accurate Specialized Task Execution for Long-Running Agents." NVIDIA Technical Blog, August 11, 2026. developer.nvidia.com/...on-for-long-running-agents
  2. ^NVIDIA AI (@NVIDIAAI). "Introducing NVIDIA Nemotron 3.5 Lightning..." X post, August 11, 2026. x.com/...2087162151995629926
  3. ^NVIDIA. "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16." Hugging Face model card, August 11, 2026. huggingface.co/...otron-3.5-Lightning-30B-A3B-BF16
  4. ^NVIDIA. "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4." Hugging Face model card, August 11, 2026. huggingface.co/...tron-3.5-Lightning-30B-A3B-NVFP4
  5. ^NVIDIA. "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark." Hugging Face repository. huggingface.co/...5-Lightning-30B-A3B-NVFP4-DSpark
  6. ^NVIDIA. "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash." Hugging Face repository. huggingface.co/...5-Lightning-30B-A3B-NVFP4-DFlash
  7. ^NVIDIA. "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16." Hugging Face repository. huggingface.co/...-3.5-Lightning-30B-A3B-Base-BF16
  8. ^OpenMDW. "OpenMDW License Agreement, version 1.1." openmdw.ai/...1-1
  9. ^NVIDIA. "Nemotron-RL-Agentic-Terminal-Pivot-v1." Hugging Face dataset, August 2026. huggingface.co/...ron-RL-Agentic-Terminal-Pivot-v1
  10. ^Tanay Varshney, Chris Alexiuk, Ayush Agarwal, Annie Surla, and Moon Chung. "Route AI Agent Workloads Across Models with NVIDIA NeMo Switchyard." NVIDIA Technical Blog, August 11, 2026. developer.nvidia.com/...ith-nvidia-nemo-switchyard
  11. ^NVIDIA-NeMo. "Switchyard." GitHub repository (Apache 2.0). github.com/...Switchyard
  12. ^Artificial Analysis. "NVIDIA launches Nemotron 3.5 Lightning." August 11, 2026. artificialanalysis.ai/...tron-3-5-lightning-launch
  13. ^Matthias Bastian. "Nvidia's open-weight Nemotron 3.5 Lightning prioritizes speed over maximum intelligence." The Decoder, August 11, 2026. the-decoder.com/...speed-over-maximum-intelligence
  14. ^Asif Razzaq. "NVIDIA AI Releases Nemotron 3.5 Lightning: A 30B Open MoE with 3B Active Parameters, and NeMo Switchyard Model Router." MarkTechPost, August 11, 2026. marktechpost.com/...-lightning-and-nemo-switchyard
  15. ^Ollama. "nemotron-3.5-lightning." Ollama model library, August 2026. ollama.com/...nemotron-3.5-lightning
  16. ^NVIDIA. "NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 model card." build.nvidia.com. build.nvidia.com/...modelcard
  17. ^NVIDIA. "NVIDIA Debuts Nemotron 3 Family of Open Models." NVIDIA Newsroom, December 15, 2025. nvidianews.nvidia.com/...n-3-family-of-open-models
  18. ^NVIDIA. "NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16." Hugging Face model card (OpenMDW-1.1), June 2026. huggingface.co/...-Nemotron-3-Ultra-550B-A55B-BF16
  19. ^NVIDIA-NeMo. "Nemotron 3.5 Lightning evaluation reproducibility recipes." NeMo Gym, GitHub. github.com/...reproducibility.md

Improve this article

Add missing citations, update stale details, or suggest a clearer explanation. Every suggestion is reviewed for sourcing before it goes live.

v1 · 3,003 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Full check on 2026-08-13: benchmarks, architecture, OpenMDW-1.1 license, DSpark/DFlash, Switchyard results, and partner names all confirmed against the NVIDIA blogs, HF model cards, license text, and

Cite this page: AI Wiki. "Nemotron 3.5 Lightning." aiwiki.ai, updated 12 Aug 2026, fact-checked 12 Aug 2026. CC BY 4.0. https://aiwiki.ai/wiki/nemotron_3_5_lightning

Suggest edit