Nemotron 3.5 Bolt

Nemotron 3.5 Bolt is an open-weight vision-language model released by NVIDIA on October 15, 2026 for local reasoning, coding, tool use, and browser and terminal tasks. It has 74 billion total parameters and 9 billion active parameters. Bolt was compressed from Nemotron 3.5 Super VL, rather than trained as an unrelated smaller model. NVIDIA released BF16 weights for customization and a mixed-precision, predominantly 2-bit version named Nemotron 3.5 Bolt 74B-A9B 24GB for inference.^[1][2]

The "24GB" in the name is a memory target, not a separate architecture. NVIDIA's 24GB estimate covers text-only serving at batch size one and a 131,072-token context, including the speculative-decoding drafter but excluding the vision encoder and projector. NVIDIA's own llama.cpp recipes budget 28GB or 32GB, depending on context length.^[3][8] NVIDIA built the model for local agents on its RTX Spark Windows PCs; NVIDIA and Microsoft said the first RTX Spark laptops would be available on October 16, 2026, the day after Bolt's release.^[14]

Model specifications

The model cards specify the architecture, modalities, and license. The context limit is shared by prompts, visual inputs, conversation history, tool results, and generated text; deployments can select a smaller window. Reasoning can be switched on or off through chat-template controls, and NVIDIA recommends a temperature of 1.0 and top-p of 0.95 for text.^[1][2][4] The license is OpenMDW-1.1, and the model cards state that the model is ready for commercial or non-commercial use.^[1]

Compression and training

Bolt retains the parent's hybrid Mamba, attention, and mixture-of-experts structure. NVIDIA used Star Elastic to select expert counts and intermediate widths under a memory budget, with knowledge distillation from the parent. The number of selected routed experts per token was reduced from 22 to nine.^[3]

The architecture router targeted a footprint below 24GB at batch size one and 128K context, with the budget already accounting for the later 2-bit quantization. Search ran at an 8K context, with the router converging during the first 8 billion tokens. Distillation then continued to 72 billion tokens at 8K and a further 48 billion tokens at 128K, for 120 billion tokens in total. NVIDIA reports that this stage used 32 GB200 nodes (128 GPUs) for about 64 hours.^[3]

Star Elastic is a research method for learning nested submodels through trainable routing and curriculum-based distillation. Its original paper studies several compression axes; Bolt's release report describes the particular architecture search applied to this model.^[5]

Because compression was applied only to the text tower, NVIDIA then re-attached the parent's vision tower and fine-tuned the vision adapter on about 6.6 million image-text samples per epoch. A vision-heavy supervised fine-tuning stage followed, using data blends aimed at ScreenSpot Pro and WebArena alongside synthetic text data generated with Kimi K3 and a small share of GLM 5.2 trajectories targeting MCP-Atlas. The model was then trained with RLHF and end-to-end reinforcement learning in MCP and software-engineering environments, with checkpoint merges (SLERP) after the fine-tuning and reinforcement learning stages. NVIDIA reports that the MCP environment raised its MCP-Atlas score from about 48 to about 56 on the 500 public tasks.^[1][3]

Quantization and memory

The local checkpoint is not uniformly 2-bit. Its routed and shared expert projections use IQ2_XXS at 2.0625 bits per element. Mamba projections use NVFP4, while attention projections, latent-MoE projections, token embeddings, and the language-model head use Q8_0. MoE router weights and Mamba convolution weights stay in FP32. NVIDIA applied quantization-aware distillation to recover accuracy after compression.^[3][6] Only the weights are quantized; NVIDIA's release page says execution on RTX and DGX Spark is mostly limited by weight bandwidth.^[6] The technical report puts the reduction at 85.1%, from a 148GB BF16 compressed model to about 22GB.^[3]

Quantization-aware distillation trains a quantized student against a full-precision teacher. The earlier research paper describes the method for NVFP4, while Bolt extends it to its lower-bit configuration. For Bolt, NVIDIA reports that a weighted Jaccard loss worked best of the losses it tried. It ran the distillation in two stages: 26 billion tokens against an earlier, high-accuracy BF16 checkpoint, then 10 million tokens against the final BF16 checkpoint to improve token efficiency. NVIDIA describes the quantization as "very aggressive and lossy by design", arguing that a larger network at 2 bits gives better accuracy for its size than a smaller one at higher precision. It does not preserve every BF16 benchmark result.^[3][7]

NVIDIA's memory estimate uses batch size one, a 131,072-token context, and an 8,192-token Mamba prefill chunk:^[3]

Released checkpoints and inference

NVIDIA supplies llama.cpp and vLLM deployment recipes.^[3] The GGUF repository ships the vision projector as a separate file (about 1.7GB) next to the 22GB language-model file, and NVIDIA's llama.cpp commands load it alongside the main model for image and video input.^[8] The GGUF model card gives two configurations:^[8]

The vLLM checkpoint lists support for NVIDIA Blackwell SM120 and SM121 GPUs. Its test hardware includes the GeForce RTX 5090, DGX Spark and RTX Spark.^[2] NVIDIA's technical report says neither runtime was ready as it stood: llama.cpp lacked LiLiCorr support and had inefficiencies in its SSM scan, and vLLM lacked efficient IQ2_XXS and Q8_0 execution on SM12x consumer GPUs. NVIDIA added kernels for those formats to the b12x library integrated into vLLM.^[3]

The LiLiCorr checkpoint is a drafter, not a standalone assistant. It proposes candidate tokens that the compatible 24GB Bolt target verifies through speculative decoding, and the target produces the final text. The NVFP4-quantized drafter has about 1.1 billion parameters, excluding shared target embeddings and the LM head.^[3][9]

LiLiCorr jointly scores candidates across a draft block to improve their coherence, rather than selecting each position independently as DFlash does. Its paper reports that, over the vanilla DFlash drafter it builds on, LiLiCorr raised acceptance length by 7 to 19% across the settings tested.^[10] On NVIDIA's AA AgentPerf Local estimate for Bolt, LiLiCorr reached an acceptance length of 4.18, compared with 3.68 for a DSpark head trained on the same backbone for the same number of steps.^[3]

Release and RTX Spark

NVIDIA announced Bolt on October 13, 2026, in a blog post about software for RTX Spark. The post called it "purpose-built for running local agents on RTX Spark" and said it would release on Thursday, October 15, ahead of the RTX Spark laptop launch on October 16. NVIDIA said users would be able to run it at launch through llama.cpp, vLLM, Ollama, Unsloth and LM Studio, and directly through agents such as Hermes and OpenClaw. It also described Bolt as fully open source, including training data, weights and recipes.^[14] On release day, NVIDIA's RTX Spark account promoted the model as "74B-A9B parameters packed into 24GB of memory" and "Optimized for RTX Spark".^[15] Ollama lists the model as nemotron-3.5-bolt, tagged for vision, tools and thinking.^[16]

NVIDIA's technical report says the experts were quantized to 2 bits "to fit the network in RTX Spark's memory", and its release page says the quantized checkpoint is optimized for RTX Spark with llama.cpp and for DGX Spark with vLLM.^[3][6]

Bolt family

Artificial Analysis described the 74B-A9B model as the first model in NVIDIA's new Bolt family.^[13] In its October 13 blog post, NVIDIA announced a second model, Nemotron 3.5 Bolt E4B 4GB, the third-generation small language model for its NVIDIA ACE game-character technology, scheduled for release on October 23, 2026; it was released that day as Nemotron 3.5 Bolt E4B, a Gemma 4 E4B-based model post-trained by NVIDIA rather than a compressed Nemotron. NVIDIA said it would be available to try in SkyrimNET, an NPC modding platform for The Elder Scrolls V: Skyrim.^[14]

Reported evaluations

NVIDIA published the following results for the BF16 and native IQ2 releases, alongside its BF16 parent. These are developer-reported evaluations, not measurements of every third-party quantization.^[1][3]

NVIDIA's MCP-Atlas evaluation used the 500 public tasks, not the full 1,000-task set. Its WebArena evaluation used screenshot-based interaction, an internal parallelized harness, and up to 100 agent steps per task. Harnesses, tool access, and precision therefore matter when comparing scores.^[3] NVIDIA's comparison tables also include Qwen3.6-35B-A3B, Gemma 4 26B-A4B and Nemotron 3.5 Lightning 30B-A3B. NVIDIA notes that it measured Qwen3.6-35B-A3B at 51.2 on MCP-Atlas against a reported 62.8 that it could not reproduce.^[1] On public leaderboards, NVIDIA's release page claims Bolt 24GB ranks first on PinchBench and second on WebArena and BrowseComp among models with a similar memory footprint.^[6]

In its October 15, 2026 evaluation, Artificial Analysis reported an Intelligence Index score of 21.4 for Nemotron 3.5 Bolt 74B-A9B. That placed it fourth among models under 100 billion parameters and first among US models in that class. The score was 8.6 points above Nemotron 3 Super (12.8) and, by Artificial Analysis's count, 1.5 points below Nemotron 3 Ultra 550B A55B (shown rounded as 23). Bolt used 82,000 output tokens per index task, 1.8 times as many as Nemotron 3.5 Super VL (46,000) and 2.9 times as many as Nemotron 3 Ultra (28,000).^[13]

Inference speed

NVIDIA reported 83.99 tokens per second on AIPerf SPEED-Bench-Coding and a 600.80-second AA AgentPerf Local replay duration for the IQ2 model with LiLiCorr. The measurements used llama.cpp on Windows 11 on Arm, on an RTX Spark system identified as a 128GB Surface Laptop Ultra. They are configuration-specific results, not guaranteed speeds on every 24GB GPU.^[3][6] In the same test, two smaller-active-parameter models ran faster than Bolt:^[6]

Unsloth GGUF distribution

Unsloth separately distributes Bolt in GGUF format. Its repository identifies NVIDIA's BF16 Bolt checkpoint as the base model and contains a range of its own quantizations, from 1-bit UD-IQ1_S (about 18.5GB) through Q8_0 and BF16, alongside an IQ2_XXS file, a vision projector and an NVFP4 DFlash drafter.^[11] These distributions should not be confused with a new NVIDIA architecture or assumed to reproduce the official checkpoint's evaluations.^[11][12]

Unsloth's memory guidance distinguishes downloadable weights from runtime requirements. It lists a 22GB download for the 2-bit IQ2_XXS quantization and suggests budgeting around 32GB of RAM or unified memory for short-context use, or 48-64GB for more headroom. Longer contexts and image or video input need extra memory for the cache and vision projector, and the LiLiCorr drafter needs about 1-2GB more. Its guide recommends beginning at a 4K context and increasing it as memory permits. The guide's 128K context setting is narrower than NVIDIA's released 256K model limit.^[2][12]

Continue reading

• Nemotron 3.5 Super VLNemotron 3.5 Super VL is an open-weight vision language model released by NVIDIA on October 15, 2026. It combines text, image, document, and video understanding with reasoning and tool use for agents.Mentioned 3 times above
• VVision language modelVision-language models (VLMs) are artificial intelligence models that learn relationships between visual data and natural language.Linked in the introduction
• NemotronNemotron is NVIDIA's brand for its family of open large language models and the datasets, training recipes, and evaluation tools built around them.Mentioned 22 times above
• NNVIDIA RTX SparkNVIDIA RTX Spark is a consumer "superchip" for Windows personal computers, announced by NVIDIA on May 31, 2026, at the company's GTC Taipei keynote held during COMPUTEX 2026.Mentioned 14 times above
• Nemotron 3.5 Bolt E4BNemotron 3.5 Bolt E4B (full name NVIDIA Nemotron 3.5 Bolt E4B 4GB) is an open small language model that NVIDIA released on October 23, 2026 for fast conversational agents: voice and device assistants…Mentioned twice above
• llama.cppllama.cpp is an open-source large language model inference engine written in C and C++ by Bulgarian software engineer Georgi Gerganov that runs large language models on consumer-grade hardware without…Mentioned 8 times above

More to read

• GGUF
• vLLM
• Mamba
• NVFP4
• Ollama
• Nemotron 3.5 Lightning

Topics:Large Language ModelsMultimodal AINVIDIAReasoning Models

References

1. ^1 ^2 ^3 ^4 ^5 ^6NVIDIA. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 model card. Released October 15, 2026.
2. ^1 ^2 ^3 ^4NVIDIA. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB model card. Released October 15, 2026.
3. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16NVIDIA. Nemotron 3.5 Bolt 74B-A9B 24GB: A Hybrid Vision-Language Model for Local AI. Technical report, October 15, 2026. Sections 1-8; Tables 1, 2, 6, 7, and 8.
4. ^NVIDIA. Bolt BF16 configuration. `llm_config.max_position_embeddings`.
5. ^Ali Taghibakhshi et al. Star Elastic: Many-in-One Reasoning LLMs with Efficient Budget Control. arXiv:2605.07182, May 8, 2026.
6. ^1 ^2 ^3 ^4 ^5 ^6NVIDIA. NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB. Research release, October 15, 2026.
7. ^Meng Xin et al. Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery. arXiv:2601.20088, January 27, 2026; revised March 3, 2026.
8. ^1 ^2 ^3NVIDIA. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF model card. Released October 15, 2026.
9. ^NVIDIA. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr model card. Released October 15, 2026.
10. ^Matan Rusanovsky et al. LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding. arXiv:2608.20530, August 20, 2026; revised September 22, 2026.
11. ^1 ^2Unsloth. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF model repository. Accessed October 17, 2026.
12. ^1 ^2Unsloth. NVIDIA Nemotron 3.5 Bolt: Run Locally Guide. Accessed October 17, 2026.
13. ^1 ^2Artificial Analysis. NVIDIA Releases Nemotron 3.5 Super and Nemotron 3.5 Bolt. October 15, 2026.
14. ^1 ^2 ^3Gerardo Delgado, NVIDIA. All Systems Go: Global PC App Ecosystem Ready for NVIDIA RTX Spark. NVIDIA Blog, October 13, 2026.
15. ^NVIDIA RTX Spark (@NVIDIARTXSpark). Post announcing Nemotron 3.5 Bolt 24GB. X, October 15, 2026.
16. ^Ollama. nemotron-3.5-bolt. Ollama model library. Accessed October 17, 2026.

Show all 16 references