Nemotron 3.5 Bolt

Nemotron 3.5 Bolt is an open-weight vision-language model released by NVIDIA on October 15, 2026 for local reasoning, coding, tool use, and browser and terminal tasks. It has 74 billion total parameters and 9 billion active parameters. Bolt was compressed from Nemotron 3.5 Super VL, rather than trained as an unrelated smaller model. NVIDIA released BF16 weights for customization and a mixed-precision, predominantly 2-bit version named Nemotron 3.5 Bolt 74B-A9B 24GB for inference.^[1][2]

NVIDIA's 24GB estimate covers text-only serving at batch size one and a 131,072-token context, excluding the vision encoder and projector. Other contexts, visual inputs, and concurrency settings have different memory requirements.^[3]

Model specifications

The model cards specify the architecture, modalities, and license. The context limit is shared by prompts, visual inputs, conversation history, tool results, and generated text; deployments can select a smaller window.^[1][2][4]

Compression and training

Bolt retains the parent's hybrid Mamba, attention, and mixture-of-experts structure. NVIDIA used Star Elastic to select expert counts and intermediate widths under a memory budget, with knowledge distillation from the parent. The number of selected routed experts per token was reduced from 22 to nine.^[3]

Star Elastic is a research method for learning nested submodels through trainable routing and curriculum-based distillation. Its original paper studies several compression axes; Bolt's release report describes the particular architecture search applied to this model.^[5]

Quantization and memory

The local checkpoint is not uniformly 2-bit. Its routed and shared expert projections use IQ2_XXS at 2.0625 bits per element. Mamba projections use NVFP4, while attention projections, latent-MoE projections, token embeddings, and the language-model head use Q8_0. NVIDIA applied quantization-aware distillation to recover accuracy after compression.^[6]

Quantization-aware distillation trains a quantized student against a full-precision teacher. The earlier research paper describes the method for NVFP4, while Bolt extends it to its lower-bit configuration. Quantization does not preserve every BF16 benchmark result.^[6][7]

NVIDIA's memory estimate uses batch size one, a 131,072-token context, and an 8,192-token Mamba prefill chunk:^[3]

Released checkpoints and inference

NVIDIA supplies llama.cpp and vLLM deployment recipes. GGUF image and video inference also requires the matching vision projector; it is not contained in the main language-model file.^[8]

The LiLiCorr checkpoint is a drafter, not a standalone assistant. It proposes candidate tokens that the compatible Bolt target verifies through speculative decoding. The target produces the final text. Its checkpoint must match the target model.^[9]

LiLiCorr jointly scores candidates across a draft block to improve their coherence, rather than selecting each position independently. The method's research results describe the tested systems and do not establish a universal speedup for every deployment.^[10]

Reported evaluations

NVIDIA published the following results for the BF16 and native IQ2 releases. These are developer-reported evaluations, not measurements of every third-party quantization.^[1]

NVIDIA's MCP-Atlas evaluation used the 500 public tasks, not the full 1,000-task set. Its WebArena evaluation used screenshot-based interaction, an internal parallelized harness, and up to 100 agent steps per task. Harnesses, tool access, and precision therefore matter when comparing scores.^[3]

In its October 15, 2026 evaluation, Artificial Analysis reported an Intelligence Index score of 21.4 for Nemotron 3.5 Bolt 74B-A9B and 82,000 output tokens per index task. These are that evaluator's aggregate score and token-use measurement, not a claim that the model is error-free.^[13]

Inference speed

NVIDIA reported 83.99 tokens per second on AIPerf SPEED-Bench-Coding and a 600.80-second AA AgentPerf Local replay duration for the IQ2 model with LiLiCorr. The measurements used llama.cpp on an RTX Spark system identified as a 128GB Surface Laptop Ultra. They are configuration-specific results, not guaranteed speeds on every 24GB GPU.^[6]

Unsloth GGUF distribution

Unsloth separately distributes Bolt in GGUF format. Its repository identifies NVIDIA's BF16 Bolt checkpoint as the base model, while its guide describes additional low-bit variants and modifications to the official IQ2 release. These distributions should not be confused with a new NVIDIA architecture or assumed to reproduce the official checkpoint's evaluations.^[11][12]

Unsloth's memory guidance distinguishes downloadable weights from runtime requirements: longer contexts, visual inputs, the projector, and a drafter need additional memory. Its guide recommends beginning at a 4K context and increasing it as memory permits. The guide's 128K context setting is narrower than NVIDIA's released 256K model limit.^[2][12]

References

1. ^1 ^2 ^3NVIDIA. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 model card. Released October 15, 2026.
2. ^1 ^2 ^3NVIDIA. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB model card. Released October 15, 2026.
3. ^1 ^2 ^3 ^4NVIDIA. Nemotron 3.5 Bolt 74B-A9B 24GB: A Hybrid Vision-Language Model for Local AI. Technical report, October 15, 2026. Sections 2-5 and 8; Tables 2, 7, and 8.
4. ^NVIDIA. Bolt BF16 configuration. `llm_config.max_position_embeddings`.
5. ^Ali Taghibakhshi et al. Star Elastic: Many-in-One Reasoning LLMs with Efficient Budget Control. arXiv:2605.07182, May 8, 2026.
6. ^1 ^2 ^3NVIDIA. NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB. Research release, October 15, 2026.
7. ^Meng Xin et al. Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery. arXiv:2601.20088, January 27, 2026; revised March 3, 2026.
8. ^NVIDIA. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF model card. Released October 15, 2026.
9. ^NVIDIA. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr model card. Released October 15, 2026.
10. ^Matan Rusanovsky et al. LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding. arXiv:2608.20530, August 20, 2026; revised September 22, 2026.
11. ^Unsloth. NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF model repository. Accessed October 16, 2026.
12. ^1 ^2Unsloth. NVIDIA Nemotron 3.5 Bolt: Run Locally Guide. Accessed October 16, 2026.
13. ^Artificial Analysis. NVIDIA Releases Nemotron 3.5 Super and Nemotron 3.5 Bolt. October 15, 2026.