# Mistral NeMo

> Source: https://aiwiki.ai/wiki/mistral_nemo
> Updated: 2026-07-24
> Fact-checked: 2026-07-24
> Categories: Large Language Models, NVIDIA, Open Source AI
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Mistral NeMo." aiwiki.ai, 24 Jul 2026. https://aiwiki.ai/wiki/mistral_nemo
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

Mistral NeMo is a 12 billion parameter [large language model](https://aiwiki.ai/wiki/large_language_model) released by [Mistral AI](https://aiwiki.ai/wiki/mistral_ai) in collaboration with [NVIDIA](https://aiwiki.ai/wiki/nvidia) on July 18, 2024 [1][2]. It shipped under the [Apache 2.0 license](https://aiwiki.ai/wiki/apache_license) in both pretrained (base) and instruction-tuned versions, offered a [context window](https://aiwiki.ai/wiki/context_window) of up to 128k tokens, and introduced Tekken, a new tokenizer based on OpenAI's [tiktoken](https://aiwiki.ai/wiki/tiktoken) that compresses text and source code more efficiently than the SentencePiece tokenizer used in earlier Mistral models [1]. Mistral positioned the model as "a drop-in replacement in any system using Mistral 7B," the 7 billion parameter release that had made the company's name [1].

The model was trained on NVIDIA's [DGX Cloud](https://aiwiki.ai/wiki/nvidia_dgx_cloud) platform using 3,072 [H100](https://aiwiki.ai/wiki/nvidia_h100) 80GB GPUs and the Megatron-LM training library, part of the [NVIDIA NeMo](https://aiwiki.ai/wiki/nvidia_nemo) platform [2]. At release, Mistral compared the base model favorably against [Gemma 2](https://aiwiki.ai/wiki/gemma_2) 9B and [Llama 3](https://aiwiki.ai/wiki/llama_3) 8B, its closest open-weight competitors in that size class at the time [1].

Beyond its own merits, Mistral NeMo mattered as raw material for other models. In August 2024 NVIDIA pruned and distilled it into Mistral-NeMo-Minitron 8B, which beat [Llama 3.1](https://aiwiki.ai/wiki/llama_3_1) 8B on a wide set of benchmarks at a fraction of the training cost of a from-scratch model [5][6]. On Mistral's own platform the model has since been superseded: the hosted `open-mistral-nemo-2407` endpoint was deprecated on May 22, 2026 and is scheduled for retirement on July 31, 2026, with Ministral 3 8B named as its replacement [10].

## Development and release

Mistral NeMo was announced simultaneously by Mistral AI and NVIDIA on July 18, 2024. NVIDIA supplied the training infrastructure and software stack: the model was trained on DGX Cloud with 3,072 H100 80GB Tensor Core GPUs using Megatron-LM, and inference was accelerated with [TensorRT-LLM](https://aiwiki.ai/wiki/tensorrt_llm) [2]. Guillaume Lample, Mistral AI cofounder and chief scientist, said of the partnership: "Together, we have developed a model with unprecedented accuracy, flexibility, high-efficiency and enterprise-grade support and security thanks to NVIDIA AI Enterprise deployment" [2].

Both checkpoints went up on [Hugging Face](https://aiwiki.ai/wiki/hugging_face) as `mistralai/Mistral-Nemo-Base-2407` and `mistralai/Mistral-Nemo-Instruct-2407`, and the model became available on Mistral's La Plateforme API as `open-mistral-nemo-2407` [1]. NVIDIA packaged it as a [NIM](https://aiwiki.ai/wiki/nvidia_nim) inference microservice on ai.nvidia.com and noted that the 12B model fits on a single NVIDIA L40S, GeForce RTX 4090, or RTX 4500 GPU [2].

The release slotted into a busy stretch for small and mid-sized open models, and Mistral's pitch rested on a longer context window than rivals of the time offered at that size, broad multilingual coverage, and a permissive license. Mistral said it released the checkpoints under Apache 2.0 "to promote adoption for researchers and enterprises" [1].

## Architecture

Mistral NeMo is a dense decoder-only transformer. Mistral emphasized that it "relies on standard architecture," which is what makes it swappable into systems built around [Mistral 7B](https://aiwiki.ai/wiki/mistral_7b) [1]. The published configuration [3]:

| Property | Value |
|---|---|
| Parameters | 12B |
| Layers | 40 |
| Model dimension | 5,120 |
| Feedforward (hidden) dimension | 14,336 |
| Attention heads | 32 |
| KV heads | 8 ([grouped-query attention](https://aiwiki.ai/wiki/grouped_query_attention)) |
| Head dimension | 128 |
| Activation | [SwiGLU](https://aiwiki.ai/wiki/swiglu) |
| Vocabulary size | 131,072 (2^17) |
| [Rotary embedding](https://aiwiki.ai/wiki/rotary_position_embedding) theta | 1,000,000 |
| Context window | 128k tokens |

The large rotary base frequency (theta of 1M, versus 10,000 in the original Mistral 7B) supports the long context window, and unlike Mistral 7B the model does not use sliding-window attention [3].

Two training details stand out. First, the model "was trained with quantisation awareness, enabling [FP8](https://aiwiki.ai/wiki/fp8) inference without any performance loss" [1]. NVIDIA's announcement framed the same property from the deployment side: the model "uses the FP8 data format for model inference, which reduces memory size and speeds deployment without any degradation to accuracy" [2]. This [quantization-aware training](https://aiwiki.ai/wiki/quantization_aware_training) claim was uncommon for open releases in mid-2024, when most models were trained in BF16 and quantized after the fact. Second, the model behaves best at unusually low sampling temperature: Mistral's model cards note that "unlike previous Mistral models, Mistral Nemo requires smaller temperatures" and recommend 0.3 [3][4].

The instruct version "underwent an advanced fine-tuning and alignment phase" and was trained for [function calling](https://aiwiki.ai/wiki/function_calling) [1][4]. Relative to Mistral 7B, Mistral reported it is "much better at following precise instructions, reasoning, handling multi-turn conversations, and generating code" [1].

## Tekken tokenizer

Mistral NeMo abandoned the SentencePiece tokenizer of earlier Mistral models for Tekken, a byte-level BPE tokenizer built on the tiktoken library and trained on more than 100 languages [1]. The switch was central to the model's multilingual positioning: a tokenizer that splits non-English text into fewer tokens makes the same context window hold more content and makes generation cheaper per unit of text.

Mistral's published comparisons: against its previous SentencePiece [tokenization](https://aiwiki.ai/wiki/tokenization), Tekken is roughly 30 percent more efficient at compressing source code, Chinese, Italian, French, German, Spanish, and Russian, about 2x more efficient for Korean, and about 3x more efficient for Arabic. Against the Llama 3 tokenizer, Mistral reported Tekken "proved more proficient in compressing text for approximately 85% of all languages" [1].

The 131,072-entry Tekken vocabulary reappeared in later Mistral releases, including the Ministral 8B checkpoint shipped that October [11].

## Performance

Mistral published base-model comparisons against Gemma 2 9B and Llama 3 8B, showing Mistral NeMo ahead on most reported benchmarks, and multilingual results across nine languages [1]. Headline numbers from the model card [3][4]:

| Benchmark | Setting | Score |
|---|---|---|
| [HellaSwag](https://aiwiki.ai/wiki/hellaswag) | 0-shot | 83.5% |
| [Winogrande](https://aiwiki.ai/wiki/winogrande) | 0-shot | 76.8% |
| [TriviaQA](https://aiwiki.ai/wiki/triviaqa) | 5-shot | 73.8% |
| CommonSenseQA | 0-shot | 70.4% |
| [MMLU](https://aiwiki.ai/wiki/mmlu) | 5-shot | 68.0% |
| [TruthfulQA](https://aiwiki.ai/wiki/truthfulqa) | 0-shot | 50.3% |

On multilingual MMLU (5-shot), the model card reports 64.6% for Spanish, 63.3% for Portuguese, 62.7% for German, 62.3% for French, 59.2% for Russian, and 59.0% for both Chinese and Japanese [3]. Mistral described the model as "particularly strong in English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Korean, Arabic, and Hindi" and said it was "designed for global, multilingual applications" [1]. Instruct-model accuracy figures were reported in the announcement with GPT-4o as judge on official references [1].

Longer-term reception treated the model less as a benchmark leader than as a workhorse. The permissive license, mid-range size, and long context made it a favorite base for community [fine-tuning](https://aiwiki.ai/wiki/fine_tuning): as of mid-2026, the Hugging Face model tree for the instruct checkpoint lists over 200 fine-tunes, nearly a hundred adapters, dozens of merges, and over 170 quantized conversions, and the checkpoint was still drawing hundreds of thousands of downloads per month [4].

## Minitron derivatives

In August 2024 NVIDIA used Mistral NeMo as the teacher model for a widely discussed compression exercise. Mistral-NeMo-Minitron 8B, announced on August 21, 2024, was produced by width-[pruning](https://aiwiki.ai/wiki/pruning) the 12B model and then retraining it with [knowledge distillation](https://aiwiki.ai/wiki/knowledge_distillation), following NVIDIA's [Minitron](https://aiwiki.ai/wiki/minitron) recipe [5][8].

The procedure, described in NVIDIA's technical write-up and an accompanying paper: the teacher model was first fine-tuned on 127B tokens of the distillation dataset to correct for distribution shift, then the network was pruned along its width, cutting the model dimension from 5,120 to 4,096 and the feedforward dimension from 14,336 to 11,520 while keeping all 40 layers and the attention head count, and finally the pruned model was distilled on 380B tokens [6][7]. NVIDIA claimed the pruning-plus-distillation approach saves "up to 40x the compute cost" compared with training a similar small model from scratch, and that the result led on nine popular benchmarks for its size [5]. Bryan Catanzaro, NVIDIA's vice president of applied deep learning research, summarized the outcome: "Mistral-NeMo-Minitron 8B delivers comparable accuracy to the original model at lower computational cost" [5].

Reported scores against Llama 3.1 8B [6]:

| Benchmark | Mistral-NeMo-Minitron 8B | Llama 3.1 8B |
|---|---|---|
| MMLU (5-shot) | 69.51 | 65.28 |
| [GSM8K](https://aiwiki.ai/wiki/gsm8k) | 58.45 | 48.60 |
| HellaSwag (10-shot) | 83.03 | 81.80 |
| Winogrande (5-shot) | 80.35 | 77.27 |
| MBPP (0-shot) | 43.77 | 42.27 |

A caution on a common misreading: the Minitron paper that covers Mistral-NeMo-Minitron 8B also produced a 4B model, but that one was compressed from Llama 3.1 8B, not from Mistral NeMo [7]. NVIDIA did not release a 4B derivative of Mistral NeMo itself; the small model it promoted alongside Minitron 8B for RTX AI PCs, Nemotron-Mini-4B-Instruct, comes from NVIDIA's own [Nemotron](https://aiwiki.ai/wiki/nemotron) line [5].

## Succession and retirement

Mistral NeMo's run as Mistral's small open model was short. In October 2024 the company introduced the Ministraux ([Ministral](https://aiwiki.ai/wiki/ministral) 3B and 8B), edge-focused models that took over the small end of its lineup, though under research and commercial licenses rather than Apache 2.0 [9]. Later small Mistral releases returned to Apache 2.0, and by 2026 the company's documentation listed Mistral NeMo among its legacy models: the `open-mistral-nemo-2407` API endpoint was deprecated on May 22, 2026, with retirement set for July 31, 2026 and Ministral 3 8B named as the migration target [10].

The [open weights](https://aiwiki.ai/wiki/open_weights) are unaffected by the API retirement. The Apache 2.0 checkpoints remain on Hugging Face, where the model continues to serve as a base for community fine-tuning, and the Minitron derivative remains available through NVIDIA [3][4][5]. The compression work outlived the model itself: the same paper that produced Mistral-NeMo-Minitron 8B applied the recipe to Meta's Llama 3.1 8B, yielding a 4B [small language model](https://aiwiki.ai/wiki/small_language_model) from the same pipeline [7].

## See also

- [Mistral AI](https://aiwiki.ai/wiki/mistral_ai)
- [Mistral 7B](https://aiwiki.ai/wiki/mistral_7b)
- [Minitron](https://aiwiki.ai/wiki/minitron)
- [Ministral](https://aiwiki.ai/wiki/ministral)
- [NVIDIA NeMo](https://aiwiki.ai/wiki/nvidia_nemo)
- [Knowledge distillation](https://aiwiki.ai/wiki/knowledge_distillation)

## References

1. Mistral AI. "Mistral NeMo." July 18, 2024. https://mistral.ai/news/mistral-nemo
2. Briski, Kari. "Mistral AI and NVIDIA Unveil Mistral NeMo 12B, a Cutting-Edge Enterprise AI Model." NVIDIA Blog, July 18, 2024. https://blogs.nvidia.com/blog/mistral-nvidia-ai-model/
3. Mistral AI. "Mistral-Nemo-Base-2407" model card. Hugging Face. https://huggingface.co/mistralai/Mistral-Nemo-Base-2407
4. Mistral AI. "Mistral-Nemo-Instruct-2407" model card. Hugging Face. https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407
5. Briski, Kari. "Lightweight Champ: NVIDIA Releases Small Language Model With State-of-the-Art Accuracy." NVIDIA Blog, August 21, 2024. https://blogs.nvidia.com/blog/mistral-nemo-minitron-8b-small-language-model/
6. NVIDIA Technical Blog. "Mistral-NeMo-Minitron 8B Foundation Model Delivers Unparalleled Accuracy." August 2024. https://developer.nvidia.com/blog/mistral-nemo-minitron-8b-foundation-model-delivers-unparalleled-accuracy/
7. Sreenivas, Sharath Turuvekere; Muralidharan, Saurav; Joshi, Raviraj; et al. "LLM Pruning and Distillation in Practice: The Minitron Approach." arXiv:2408.11796, August 2024. https://arxiv.org/abs/2408.11796
8. Muralidharan, Saurav; et al. "Compact Language Models via Pruning and Knowledge Distillation." arXiv:2407.14679, July 2024. https://arxiv.org/abs/2407.14679
9. Mistral AI. "Un Ministral, des Ministraux." October 16, 2024. https://mistral.ai/news/ministraux
10. Mistral AI. "Models Overview." Mistral AI documentation (accessed July 2026). https://docs.mistral.ai/getting-started/models/models_overview/
11. Mistral AI. "Ministral-8B-Instruct-2410" model card. Hugging Face. https://huggingface.co/mistralai/Ministral-8B-Instruct-2410