# Nemotron 3.5 Bolt

> Source: https://aiwiki.ai/wiki/nemotron_3_5_bolt
> Updated: 2026-10-15
> Categories: AI Models, Large Language Models, Multimodal AI, NVIDIA, Reasoning Models
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Nemotron 3.5 Bolt." aiwiki.ai, 15 Oct 2026. https://aiwiki.ai/wiki/nemotron_3_5_bolt
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**Nemotron 3.5 Bolt** is an open-weight [vision-language model](https://aiwiki.ai/wiki/vision_language_model) released by NVIDIA on October 15, 2026 for local reasoning, coding, tool use, and browser and terminal tasks. It has 74 billion total parameters and 9 billion active parameters. Bolt was compressed from [Nemotron 3.5 Super VL](https://aiwiki.ai/wiki/nemotron_3_5_super_vl), rather than trained as an unrelated smaller model. NVIDIA released BF16 weights for customization and a mixed-precision, predominantly 2-bit version named Nemotron 3.5 Bolt 74B-A9B 24GB for inference.[1][2]

NVIDIA's 24GB estimate covers text-only serving at batch size one and a 131,072-token context, excluding the vision encoder and projector. Other contexts, visual inputs, and concurrency settings have different memory requirements.[3]

## Model specifications

| Property | Specification |
| --- | --- |
| Developer | NVIDIA |
| Model family | [Nemotron](https://aiwiki.ai/wiki/nemotron) |
| Release date | 2026-10-15 |
| Parameters | 74B total, 9B active |
| Architecture | Mamba-2 and Transformer hybrid with LatentMoE |
| Vision encoder | C-RADIO v4-H |
| Inputs | Text, images, video |
| Output | Text |
| Maximum context | 256K tokens; released configuration sets 262,144 |
| Full-precision checkpoint | BF16 |
| Local-inference checkpoint | Mixed precision, with IQ2_XXS expert weights |
| License | OpenMDW-1.1 |

The model cards specify the architecture, modalities, and license. The context limit is shared by prompts, visual inputs, conversation history, tool results, and generated text; deployments can select a smaller window.[1][2][4]

## Compression and training

Bolt retains the parent's hybrid [Mamba](https://aiwiki.ai/wiki/mamba), attention, and [mixture-of-experts](https://aiwiki.ai/wiki/mixture_of_experts) structure. NVIDIA used Star Elastic to select expert counts and intermediate widths under a memory budget, with [knowledge distillation](https://aiwiki.ai/wiki/knowledge_distillation) from the parent. The number of selected routed experts per token was reduced from 22 to nine.[3]

Star Elastic is a research method for learning nested submodels through trainable routing and curriculum-based distillation. Its original paper studies several compression axes; Bolt's release report describes the particular architecture search applied to this model.[5]

## Quantization and memory

The local checkpoint is not uniformly 2-bit. Its routed and shared expert projections use IQ2_XXS at 2.0625 bits per element. Mamba projections use NVFP4, while attention projections, latent-MoE projections, token embeddings, and the language-model head use Q8_0. NVIDIA applied quantization-aware distillation to recover accuracy after compression.[6]

Quantization-aware distillation trains a quantized student against a full-precision teacher. The earlier research paper describes the method for NVFP4, while Bolt extends it to its lower-bit configuration. Quantization does not preserve every BF16 benchmark result.[6][7]

NVIDIA's memory estimate uses batch size one, a 131,072-token context, and an 8,192-token Mamba prefill chunk:[3]

| Component | Estimated memory |
| --- | ---: |
| Model weights | 22.01 GB |
| Caches and activation buffer | 0.87 GB |
| Speculative decoder | 0.96 GB |
| Total excluding vision | 23.83 GB |
| Vision encoder and projector, listed separately | 1.68 GB |

## Released checkpoints and inference

| Release | Purpose |
| --- | --- |
| NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 | Customization, post-training, and full-precision evaluation |
| NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB | Packed local-inference weights for vLLM |
| NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF | Local-inference weights for llama.cpp |
| NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr | Companion speculative-decoding checkpoint |
| NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr-GGUF | GGUF companion drafter |

NVIDIA supplies [llama.cpp](https://aiwiki.ai/wiki/llama_cpp) and [vLLM](https://aiwiki.ai/wiki/vllm) deployment recipes. GGUF image and video inference also requires the matching vision projector; it is not contained in the main language-model file.[8]

The LiLiCorr checkpoint is a drafter, not a standalone assistant. It proposes candidate tokens that the compatible Bolt target verifies through [speculative decoding](https://aiwiki.ai/wiki/speculative_decoding). The target produces the final text. Its checkpoint must match the target model.[9]

LiLiCorr jointly scores candidates across a draft block to improve their coherence, rather than selecting each position independently. The method's research results describe the tested systems and do not establish a universal speedup for every deployment.[10]

## Reported evaluations

NVIDIA published the following results for the BF16 and native IQ2 releases. These are developer-reported evaluations, not measurements of every third-party quantization.[1]

| Benchmark | BF16 | 24GB IQ2 |
| --- | ---: | ---: |
| GDPVal-AA-V2 | 31.13 | 28.47 |
| Terminal-Bench 2.1 | 53.93 | 50.98 |
| Tau3 Banking | 17.01 | 17.01 |
| HLE, no tools | 28.73 | 25.30 |
| AA-Omniscience accuracy | 21.80 | 16.90 |
| SciCode | 44.49 | 41.02 |
| GPQA Diamond | 88.83 | 86.43 |
| AA-LCR | 67.38 | 64.06 |
| AA-Omniscience non-hallucination | 56.14 | 61.27 |
| PinchBench | 90.35 | 87.82 |
| MCP-Atlas | 55.60 | 52.20 |
| BrowseComp | 70.00 | 66.75 |
| WebArena, vision | 50.25 | 50.86 |

NVIDIA's MCP-Atlas evaluation used the 500 public tasks, not the full 1,000-task set. Its WebArena evaluation used screenshot-based interaction, an internal parallelized harness, and up to 100 agent steps per task. Harnesses, tool access, and precision therefore matter when comparing scores.[3]

In its October 15, 2026 evaluation, Artificial Analysis reported an Intelligence Index score of 21.4 for Nemotron 3.5 Bolt 74B-A9B and 82,000 output tokens per index task. These are that evaluator's aggregate score and token-use measurement, not a claim that the model is error-free.[13]

### Inference speed

NVIDIA reported 83.99 tokens per second on AIPerf SPEED-Bench-Coding and a 600.80-second AA AgentPerf Local replay duration for the IQ2 model with LiLiCorr. The measurements used llama.cpp on an RTX Spark system identified as a 128GB Surface Laptop Ultra. They are configuration-specific results, not guaranteed speeds on every 24GB GPU.[6]

## Unsloth GGUF distribution

[Unsloth](https://aiwiki.ai/wiki/unsloth) separately distributes Bolt in [GGUF](https://aiwiki.ai/wiki/gguf) format. Its repository identifies NVIDIA's BF16 Bolt checkpoint as the base model, while its guide describes additional low-bit variants and modifications to the official IQ2 release. These distributions should not be confused with a new NVIDIA architecture or assumed to reproduce the official checkpoint's evaluations.[11][12]

Unsloth's memory guidance distinguishes downloadable weights from runtime requirements: longer contexts, visual inputs, the projector, and a drafter need additional memory. Its guide recommends beginning at a 4K context and increasing it as memory permits. The guide's 128K context setting is narrower than NVIDIA's released 256K model limit.[2][12]

## References

1. NVIDIA. [NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16). Released October 15, 2026.
2. NVIDIA. [NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB). Released October 15, 2026.
3. NVIDIA. [Nemotron 3.5 Bolt 74B-A9B 24GB: A Hybrid Vision-Language Model for Local AI](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-Technical-Report.pdf). Technical report, October 15, 2026. Sections 2-5 and 8; Tables 2, 7, and 8.
4. NVIDIA. [Bolt BF16 configuration](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16/blob/main/config.json). `llm_config.max_position_embeddings`.
5. Ali Taghibakhshi et al. [Star Elastic: Many-in-One Reasoning LLMs with Efficient Budget Control](https://arxiv.org/abs/2605.07182). arXiv:2605.07182, May 8, 2026.
6. NVIDIA. [NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB](https://research.nvidia.com/labs/nemotron/Nemotron-3.5-Bolt-24GB/). Research release, October 15, 2026.
7. Meng Xin et al. [Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery](https://arxiv.org/abs/2601.20088). arXiv:2601.20088, January 27, 2026; revised March 3, 2026.
8. NVIDIA. [NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF). Released October 15, 2026.
9. NVIDIA. [NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr). Released October 15, 2026.
10. Matan Rusanovsky et al. [LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding](https://arxiv.org/abs/2608.20530). arXiv:2608.20530, August 20, 2026; revised September 22, 2026.
11. Unsloth. [NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF model repository](https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF). Accessed October 16, 2026.
12. Unsloth. [NVIDIA Nemotron 3.5 Bolt: Run Locally Guide](https://unsloth.ai/docs/models/nemotron-3.5-bolt). Accessed October 16, 2026.
13. Artificial Analysis. [NVIDIA Releases Nemotron 3.5 Super and Nemotron 3.5 Bolt](https://artificialanalysis.ai/articles/nemotron-3-5-super-and-bolt). October 15, 2026.