Best Small Language Models

As of July 2026, the two strongest small language models (open models under 15 billion parameters) are Google DeepMind's Gemma 4 12B and Alibaba's Qwen3.5-9B. On the independent Artificial Analysis Intelligence Index, Gemma 4 12B is the highest-scoring sub-15B open model, and it also leads its weight class on coding with 72.0 on LiveCodeBench; Qwen3.5-9B leads on knowledge and science reasoning, scoring 82.5 on MMLU-Pro and 81.7 on GPQA Diamond from just 9B parameters ^[1][2][4]. Both ship under Apache-2.0, are natively multimodal, and support a 256K-token context, so for most edge-deployment and fine-tuning projects the pick is Gemma 4 12B for the best all-around capability or Qwen3.5-9B for the highest reasoning-per-parameter. This is the ranked "best" list; for the concept, architecture, and history of the category, see small language models.

Quick verdict by use case:

• Best overall small model: Gemma 4 12B, with Qwen3.5-9B a close second ^[1].
• Best tiny model for edge and mobile (under 4B): Gemma 4 E2B and E4B for offline text, image, and audio; Ministral 3 3B for capability density ^[2][10].
• Best for fine-tuning: Qwen3.5-4B for a permissive, high-capability base, and SmolLM3 3B for a fully open data-and-recipe base ^[5][8].
• Best coding SLM: Gemma 4 12B (LiveCodeBench 72.0), with Qwen3.5-9B close behind (65.6) ^[2][4].
• Best multilingual SLM: Gemma 4 (140+ languages) and Qwen3.5 (119+ languages) ^[3][4].

Summary comparison table

The table ranks the leading sub-15B open models by capability-per-parameter. Last verified: July 2026. Scores are the developers' reported results from official model cards or the Artificial Analysis leaderboard; reasoning-capable models are scored in their default thinking mode. A blank cell is n/r (not reported), never 0. Parameters are total unless marked "eff." (Gemma effective size).

Scroll sideways for more →

Notes: the AIME year varies by publisher (Gemma AIME 2026; Ministral 3 and SmolLM3 AIME 2025); Qwen reports HMMT rather than AIME, with Qwen3.5-9B scoring 83.2 on HMMT Feb 2025 ^[4]. † Phi figures are the GPQA main set, not GPQA Diamond, so they are not directly comparable to the Diamond scores above ^[6][7]. Ministral 3 3B is 3.4B language parameters plus a 0.4B vision encoder ^[10].

Gemma 4 12B: best overall small model

Gemma 4 12B, the "unified" mid-size member of Google DeepMind's Gemma 4 family added in mid-2026, is the highest-ranked sub-15B model on the Artificial Analysis Intelligence Index ^[1]. It posts 77.2 on MMLU-Pro, 78.8 on GPQA Diamond, and a class-leading 72.0 on LiveCodeBench, and it is natively multimodal across text, image, and audio with a 256K context window ^[2]. Gemma 4 also switched to a true Apache-2.0 license, dropping the custom Gemma terms used through Gemma 3 and removing the prior commercial-use carve-outs ^[12]. Its 26B-A4B and 31B siblings are stronger still but exceed the sub-15B scope of this list. Best for: a single on-device or single-GPU model that has to do a bit of everything well.

Qwen3.5-9B: best reasoning per parameter

Qwen3.5-9B is the strongest sub-15B model on pure knowledge and reasoning, scoring 82.5 on MMLU-Pro and 81.7 on GPQA Diamond in its default thinking mode, ahead of every other model in its class and outscoring some open models more than ten times its size ^[4]. It also handles code well (65.6 on LiveCodeBench v6), is multimodal, and offers a 256K native context extensible toward 1M tokens, all under Apache-2.0 ^[4]. Best for: math, science, and agentic reasoning where you want frontier-adjacent quality on a 9B footprint.

Qwen3.5-4B: best sub-5B all-rounder and fine-tuning base

Qwen3.5-4B delivers 79.1 MMLU-Pro and 76.2 GPQA Diamond, a level of capability that a year earlier required models many times larger, in a 4B package with a 256K context and Apache-2.0 licensing ^[5]. Its small size, permissive license, and large tooling ecosystem make it the pragmatic default base for custom fine-tunes. Best for: fine-tuning a capable, commercially usable model that still fits comfortably on a single consumer GPU.

Gemma 4 E4B and E2B: best offline multimodal for phones and edge

The Gemma 4 "E" (effective) variants are built for on-device use. E4B (4B effective) scores 69.4 MMLU-Pro and 58.6 GPQA Diamond, and E2B (2B effective) still reaches 60.0 MMLU-Pro, both with a 128K context and text, image, and audio input that runs fully offline on phones, Raspberry Pi, and Jetson-class boards ^[2][3]. They succeed the earlier on-device Gemma 3n line. Best for: real-time, private, offline multimodal inference on constrained hardware.

Phi-4: best for math and dense knowledge

Phi-4, Microsoft's 14B dense model, remains a knowledge and math standout, scoring 84.8 on MMLU, 70.4 on MMLU-Pro, 80.4 on MATH, and 82.6 on HumanEval, the payoff of Microsoft's synthetic "textbook-quality" data recipe ^[6]. Its main limitations are a short 16K context window and text-only, non-reasoning-by-default operation, so it trades flexibility for per-parameter knowledge density. It ships under the permissive MIT license. Best for: math, structured reasoning, and knowledge tasks that do not need a long context.

Ministral 3: best multilingual, token-efficient edge family

Ministral 3, released by Mistral AI in December 2025 in 3B, 8B, and 14B dense sizes with base, instruct, and reasoning variants, targets efficient edge and multilingual deployment across 40+ languages with a 256K context, all under Apache-2.0 ^[9]. The 3B reasoning variant reaches 72.1 on AIME 2025 and 53.4 on GPQA Diamond, and Mistral emphasizes that Ministral models often emit an order of magnitude fewer tokens than rivals for the same task, which lowers latency and cost ^[10]. Best for: cost-sensitive, multilingual, latency-bound edge and on-prem workloads.

SmolLM3-3B: best fully-open model for research and fine-tuning

SmolLM3, from Hugging Face, is a 3B dual-mode reasoner with a 128K context, six-language support, and Apache-2.0 weights, but its differentiator is full openness: the training data, curriculum, and recipe are published, not just the weights ^[8]. In reasoning mode it scores 41.7 GPQA Diamond, 36.7 AIME 2025, and 30.0 LiveCodeBench, competitive with other 3-4B models ^[8]. Best for: reproducible research and fine-tunes where an auditable, fully open pipeline matters more than peak scores. A comparable fully open alternative is OLMo 2.

Phi-4-mini: compact math and reasoning workhorse

Phi-4-mini packs Microsoft's phi recipe into 3.8B parameters with a 128K context and MIT license, scoring 67.3 MMLU, 88.6 on GSM8K, and 64.0 on MATH, strong grade-school and high-school math for its size, though its GPQA is low (25.2 on the main set) ^[7]. Best for: on-device math, tool-calling, and reasoning where MIT licensing and Microsoft ecosystem support are priorities.

Which is the best small model for coding?

For code, Gemma 4 12B leads the sub-15B field at 72.0 on LiveCodeBench, with Qwen3.5-9B close behind at 65.6 ^[2][4]. Dedicated small code models such as Qwen3-Coder, Qwen2.5-Coder, and StarCoder remain useful for repository-scale and fill-in-the-middle work, but the strongest general reasoners now match or beat them on LiveCodeBench. Note that agentic SWE-bench Verified scores stay low for every model under 15B; repo-scale coding still favors 24B-and-larger models such as Devstral.

Which is the best small model for edge and on-device use (under 4B)?

Under 4B, the pick depends on the constraint. For offline multimodal input on a phone, Gemma 4 E2B (2B effective, 60.0 MMLU-Pro) is the strongest option ^[2]. For maximum reasoning in a tiny footprint, Ministral 3 3B (72.1 on AIME 2025) and Qwen3.5-2B lead, while Phi-4-mini is best for math and SmolLM3 is best when you need a fully open base ^[7][8][10].

Which small model is best for fine-tuning?

For fine-tuning, favor permissive licenses and strong bases. Qwen3.5-4B and Qwen3.5-9B (Apache-2.0, top-of-class capability, large tooling ecosystem) are the pragmatic choices, while SmolLM3 and OLMo 2 are best when a fully open, auditable data pipeline is required ^[5][8]. Gemma 4's move to Apache-2.0 also makes its E4B and 12B checkpoints clean bases for commercial fine-tunes ^[12].

Which small model is best for multilingual tasks?

Gemma 4 (140+ languages) and Qwen3.5 (119+ languages) are the broadest multilingual small models, and both post strong multilingual benchmark results ^[3][4]. Ministral 3 is a strong European-and-global option at 40+ languages with high token efficiency ^[9].

Does Llama 4 Scout qualify as a small model?

No. Llama 4 Scout is a mixture-of-experts model with 17B active parameters and 109B total parameters across 16 experts, so both its active and its total counts exceed the sub-15B bar, and all 109B must be loaded into memory ^[11]. It is an efficient mid-size open model, not a small one. For genuinely small Meta models, the older Llama 3.2 1B and 3B remain the edge options.

How these models were ranked

Rankings use each model's officially reported benchmarks (MMLU-Pro, GPQA Diamond, LiveCodeBench, and AIME) from primary model cards, cross-checked against the independent Artificial Analysis Intelligence Index, and weigh capability-per-parameter, license permissiveness, context length, and multimodality for fine-tuning and edge use ^[1][2][4]. Other sub-15B families worth tracking include IBM Granite 4.0, NVIDIA's Nemotron small models, ServiceNow's Apriel, and LG's EXAONE. Last verified: July 2026.

References

1. Artificial Analysis. "Small open-source models: Intelligence Index." https://artificialanalysis.ai/models/open-source/small ↩
2. Hugging Face and Google DeepMind. "Welcome Gemma 4: Frontier multimodal intelligence on device." https://huggingface.co/blog/gemma4 ↩
3. Google AI for Developers. "Gemma 4 model overview." https://ai.google.dev/gemma/docs/core ↩
4. Alibaba / Qwen. "Qwen3.5-9B model card." https://huggingface.co/Qwen/Qwen3.5-9B ↩
5. Alibaba / Qwen. "Qwen3.5-4B model card." https://huggingface.co/Qwen/Qwen3.5-4B ↩
6. Microsoft. "Phi-4 model card." https://huggingface.co/microsoft/phi-4 ↩
7. Microsoft. "Phi-4-mini-instruct model card." https://huggingface.co/microsoft/Phi-4-mini-instruct ↩
8. Hugging Face. "SmolLM3: smol, multilingual, long-context reasoner." https://huggingface.co/blog/smollm3 ↩
9. Mistral AI. "Introducing Mistral 3." https://mistral.ai/news/mistral-3/ ↩
10. Mistral AI. "Ministral 3 3B Instruct 2512 model card." https://huggingface.co/mistralai/Ministral-3-3B-Instruct-2512 ↩
11. Meta. "The Llama 4 herd: the beginning of a new era of natively multimodal AI innovation." https://ai.meta.com/blog/llama-4-multimodal-intelligence/ ↩
12. gHacks and Google. "Google releases Gemma 4 under Apache 2.0 license" and Gemma Terms. https://www.ghacks.net/2026/04/06/google-releases-gemma-4-in-four-model-sizes-under-apache-2-0-license/ ↩