Mistral NeMo is a 12 billion parameter large language model released by Mistral AI in collaboration with NVIDIA on July 18, 2024 [1][2]. It shipped under the Apache 2.0 license in both pretrained (base) and instruction-tuned versions, offered a context window of up to 128k tokens, and introduced Tekken, a new tokenizer based on OpenAI's tiktoken that compresses text and source code more efficiently than the SentencePiece tokenizer used in earlier Mistral models [1]. Mistral positioned the model as "a drop-in replacement in any system using Mistral 7B," the 7 billion parameter release that had made the company's name [1].

The model was trained on NVIDIA's DGX Cloud platform using 3,072 H100 80GB GPUs and the Megatron-LM training library, part of the NVIDIA NeMo platform [2]. At release, Mistral compared the base model favorably against Gemma 2 9B and Llama 3 8B, its closest open-weight competitors in that size class at the time [1].

Beyond its own merits, Mistral NeMo mattered as raw material for other models. In August 2024 NVIDIA pruned and distilled it into Mistral-NeMo-Minitron 8B, which beat Llama 3.1 8B on a wide set of benchmarks at a fraction of the training cost of a from-scratch model [5][6]. On Mistral's own platform the model has since been superseded: the hosted open-mistral-nemo-2407 endpoint was deprecated on May 22, 2026 and is scheduled for retirement on July 31, 2026, with Ministral 3 8B named as its replacement [10].

Development and release

Mistral NeMo was announced simultaneously by Mistral AI and NVIDIA on July 18, 2024. NVIDIA supplied the training infrastructure and software stack: the model was trained on DGX Cloud with 3,072 H100 80GB Tensor Core GPUs using Megatron-LM, and inference was accelerated with TensorRT-LLM [2]. Guillaume Lample, Mistral AI cofounder and chief scientist, said of the partnership: "Together, we have developed a model with unprecedented accuracy, flexibility, high-efficiency and enterprise-grade support and security thanks to NVIDIA AI Enterprise deployment" [2].

Both checkpoints went up on Hugging Face as mistralai/Mistral-Nemo-Base-2407 and mistralai/Mistral-Nemo-Instruct-2407, and the model became available on Mistral's La Plateforme API as open-mistral-nemo-2407 [1]. NVIDIA packaged it as a NIM inference microservice on ai.nvidia.com and noted that the 12B model fits on a single NVIDIA L40S, GeForce RTX 4090, or RTX 4500 GPU [2].

The release slotted into a busy stretch for small and mid-sized open models, and Mistral's pitch rested on a longer context window than rivals of the time offered at that size, broad multilingual coverage, and a permissive license. Mistral said it released the checkpoints under Apache 2.0 "to promote adoption for researchers and enterprises" [1].

Architecture

Mistral NeMo is a dense decoder-only transformer. Mistral emphasized that it "relies on standard architecture," which is what makes it swappable into systems built around Mistral 7B [1]. The published configuration [3]:

Property	Value
Parameters	12B
Layers	40
Model dimension	5,120
Feedforward (hidden) dimension	14,336
Attention heads	32
KV heads	8 (grouped-query attention)
Head dimension	128
Activation	SwiGLU
Vocabulary size	131,072 (2^17)
Rotary embedding theta	1,000,000
Context window	128k tokens

The large rotary base frequency (theta of 1M, versus 10,000 in the original Mistral 7B) supports the long context window, and unlike Mistral 7B the model does not use sliding-window attention [3].

Two training details stand out. First, the model "was trained with quantisation awareness, enabling FP8 inference without any performance loss" [1]. NVIDIA's announcement framed the same property from the deployment side: the model "uses the FP8 data format for model inference, which reduces memory size and speeds deployment without any degradation to accuracy" [2]. This quantization-aware training claim was uncommon for open releases in mid-2024, when most models were trained in BF16 and quantized after the fact. Second, the model behaves best at unusually low sampling temperature: Mistral's model cards note that "unlike previous Mistral models, Mistral Nemo requires smaller temperatures" and recommend 0.3 [3][4].

The instruct version "underwent an advanced fine-tuning and alignment phase" and was trained for function calling [1][4]. Relative to Mistral 7B, Mistral reported it is "much better at following precise instructions, reasoning, handling multi-turn conversations, and generating code" [1].

Tekken tokenizer

Mistral NeMo abandoned the SentencePiece tokenizer of earlier Mistral models for Tekken, a byte-level BPE tokenizer built on the tiktoken library and trained on more than 100 languages [1]. The switch was central to the model's multilingual positioning: a tokenizer that splits non-English text into fewer tokens makes the same context window hold more content and makes generation cheaper per unit of text.

Mistral's published comparisons: against its previous SentencePiece tokenization, Tekken is roughly 30 percent more efficient at compressing source code, Chinese, Italian, French, German, Spanish, and Russian, about 2x more efficient for Korean, and about 3x more efficient for Arabic. Against the Llama 3 tokenizer, Mistral reported Tekken "proved more proficient in compressing text for approximately 85% of all languages" [1].

The 131,072-entry Tekken vocabulary reappeared in later Mistral releases, including the Ministral 8B checkpoint shipped that October [11].

Performance

Mistral published base-model comparisons against Gemma 2 9B and Llama 3 8B, showing Mistral NeMo ahead on most reported benchmarks, and multilingual results across nine languages [1]. Headline numbers from the model card [3][4]:

Benchmark	Setting	Score
HellaSwag	0-shot	83.5%
Winogrande	0-shot	76.8%
TriviaQA	5-shot	73.8%
CommonSenseQA	0-shot	70.4%
MMLU	5-shot	68.0%
TruthfulQA	0-shot	50.3%

On multilingual MMLU (5-shot), the model card reports 64.6% for Spanish, 63.3% for Portuguese, 62.7% for German, 62.3% for French, 59.2% for Russian, and 59.0% for both Chinese and Japanese [3]. Mistral described the model as "particularly strong in English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Korean, Arabic, and Hindi" and said it was "designed for global, multilingual applications" [1]. Instruct-model accuracy figures were reported in the announcement with GPT-4o as judge on official references [1].

Longer-term reception treated the model less as a benchmark leader than as a workhorse. The permissive license, mid-range size, and long context made it a favorite base for community fine-tuning: as of mid-2026, the Hugging Face model tree for the instruct checkpoint lists over 200 fine-tunes, nearly a hundred adapters, dozens of merges, and over 170 quantized conversions, and the checkpoint was still drawing hundreds of thousands of downloads per month [4].

Minitron derivatives

In August 2024 NVIDIA used Mistral NeMo as the teacher model for a widely discussed compression exercise. Mistral-NeMo-Minitron 8B, announced on August 21, 2024, was produced by width-pruning the 12B model and then retraining it with knowledge distillation, following NVIDIA's Minitron recipe [5][8].

The procedure, described in NVIDIA's technical write-up and an accompanying paper: the teacher model was first fine-tuned on 127B tokens of the distillation dataset to correct for distribution shift, then the network was pruned along its width, cutting the model dimension from 5,120 to 4,096 and the feedforward dimension from 14,336 to 11,520 while keeping all 40 layers and the attention head count, and finally the pruned model was distilled on 380B tokens [6][7]. NVIDIA claimed the pruning-plus-distillation approach saves "up to 40x the compute cost" compared with training a similar small model from scratch, and that the result led on nine popular benchmarks for its size [5]. Bryan Catanzaro, NVIDIA's vice president of applied deep learning research, summarized the outcome: "Mistral-NeMo-Minitron 8B delivers comparable accuracy to the original model at lower computational cost" [5].

Reported scores against Llama 3.1 8B [6]:

Benchmark	Mistral-NeMo-Minitron 8B	Llama 3.1 8B
MMLU (5-shot)	69.51	65.28
GSM8K	58.45	48.60
HellaSwag (10-shot)	83.03	81.80
Winogrande (5-shot)	80.35	77.27
MBPP (0-shot)	43.77	42.27

A caution on a common misreading: the Minitron paper that covers Mistral-NeMo-Minitron 8B also produced a 4B model, but that one was compressed from Llama 3.1 8B, not from Mistral NeMo [7]. NVIDIA did not release a 4B derivative of Mistral NeMo itself; the small model it promoted alongside Minitron 8B for RTX AI PCs, Nemotron-Mini-4B-Instruct, comes from NVIDIA's own Nemotron line [5].

Succession and retirement

Mistral NeMo's run as Mistral's small open model was short. In October 2024 the company introduced the Ministraux (Ministral 3B and 8B), edge-focused models that took over the small end of its lineup, though under research and commercial licenses rather than Apache 2.0 [9]. Later small Mistral releases returned to Apache 2.0, and by 2026 the company's documentation listed Mistral NeMo among its legacy models: the open-mistral-nemo-2407 API endpoint was deprecated on May 22, 2026, with retirement set for July 31, 2026 and Ministral 3 8B named as the migration target [10].

The open weights are unaffected by the API retirement. The Apache 2.0 checkpoints remain on Hugging Face, where the model continues to serve as a base for community fine-tuning, and the Minitron derivative remains available through NVIDIA [3][4][5]. The compression work outlived the model itself: the same paper that produced Mistral-NeMo-Minitron 8B applied the recipe to Meta's Llama 3.1 8B, yielding a 4B small language model from the same pipeline [7].

References

Mistral AI. "Mistral NeMo." July 18, 2024. https://mistral.ai/news/mistral-nemo
Briski, Kari. "Mistral AI and NVIDIA Unveil Mistral NeMo 12B, a Cutting-Edge Enterprise AI Model." NVIDIA Blog, July 18, 2024. https://blogs.nvidia.com/blog/mistral-nvidia-ai-model/
Mistral AI. "Mistral-Nemo-Base-2407" model card. Hugging Face. https://huggingface.co/mistralai/Mistral-Nemo-Base-2407
Mistral AI. "Mistral-Nemo-Instruct-2407" model card. Hugging Face. https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407
Briski, Kari. "Lightweight Champ: NVIDIA Releases Small Language Model With State-of-the-Art Accuracy." NVIDIA Blog, August 21, 2024. https://blogs.nvidia.com/blog/mistral-nemo-minitron-8b-small-language-model/
NVIDIA Technical Blog. "Mistral-NeMo-Minitron 8B Foundation Model Delivers Unparalleled Accuracy." August 2024. https://developer.nvidia.com/blog/mistral-nemo-minitron-8b-foundation-model-delivers-unparalleled-accuracy/
Sreenivas, Sharath Turuvekere; Muralidharan, Saurav; Joshi, Raviraj; et al. "LLM Pruning and Distillation in Practice: The Minitron Approach." arXiv:2408.11796, August 2024. https://arxiv.org/abs/2408.11796
Muralidharan, Saurav; et al. "Compact Language Models via Pruning and Knowledge Distillation." arXiv:2407.14679, July 2024. https://arxiv.org/abs/2407.14679
Mistral AI. "Un Ministral, des Ministraux." October 16, 2024. https://mistral.ai/news/ministraux
Mistral AI. "Models Overview." Mistral AI documentation (accessed July 2026). https://docs.mistral.ai/getting-started/models/models_overview/
Mistral AI. "Ministral-8B-Instruct-2410" model card. Hugging Face. https://huggingface.co/mistralai/Ministral-8B-Instruct-2410

Editing: Mistral NeMo