# Llama 4 Scout and Maverick

> Source: https://aiwiki.ai/wiki/llama_4_scout_maverick
> Updated: 2026-06-25
> Categories: AI Models, Large Language Models, Meta AI, Mixture of Experts, Multimodal AI, Open Source AI
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution to "AI Wiki (aiwiki.ai)".

**Llama 4 Scout** and **Llama 4 Maverick** are open-weight, natively [multimodal AI](/wiki/multimodal_ai) large language models developed by [Meta](/wiki/meta_ai) and released on April 5, 2025.[1] They are the first two publicly available members of the [Llama 4](/wiki/llama_4) family, which Meta announced as "the beginning of a new era of natively multimodal AI innovation," alongside a much larger [Llama 4 Behemoth](/wiki/llama_4) model that was still in training.[1] Scout and Maverick are the first open-weight models to combine a [Mixture of Experts](/wiki/mixture_of_experts) (MoE) architecture with native multimodal capabilities through early fusion, processing both text and image inputs from the ground up.[1][2]

Llama 4 Scout is a 17-billion active parameter model with 16 experts and roughly 109 billion total parameters, and it offers a 10-million token context window while fitting on a single NVIDIA H100 GPU.[1] Llama 4 Maverick has 17 billion active parameters, 128 experts, and approximately 400 billion total parameters, with a 1-million token context window.[1] Both models share the same 17-billion active-parameter budget per token, were trained jointly on text, image, and video data, and were distilled in part from Llama 4 Behemoth, a teacher model with about 288 billion active parameters and close to two trillion total parameters.[1]

The release was accompanied by controversy after it emerged that the version Meta submitted to the LMArena chatbot leaderboard was a specially optimized experimental variant (labeled Llama-4-Maverick-03-26-Experimental) that was not available to the public and performed significantly better on that benchmark than the publicly released model.[5][6]

## What are Llama 4 Scout and Maverick?

Llama 4 Scout and Llama 4 Maverick are the two open-weight foundation models that Meta shipped first in the Llama 4 generation. Both are auto-regressive Mixture of Experts transformers that activate 17 billion parameters per token while holding far larger total parameter pools (about 109 billion for Scout and about 400 billion for Maverick), and both are natively multimodal, accepting interleaved text and image inputs.[1][2] Meta describes Scout as "the best multimodal model in the world in its class" and Maverick as "the best multimodal model in its class, beating GPT-4o and Gemini 2.0 Flash across a broad range of widely reported benchmarks."[1]

The table below summarizes the headline specifications for the three announced Llama 4 models.

| Model | Active params | Experts | Total params | Context window | Status at launch |
|---|---|---|---|---|---|
| Llama 4 Scout | 17B | 16 | ~109B | 10M | Released (open weight) |
| Llama 4 Maverick | 17B | 128 | ~400B | 1M | Released (open weight) |
| Llama 4 Behemoth | 288B | 16 | ~2T | N/A | Preview, still in training |

## Background

Meta has released successive generations of open-weight language models under the [LLaMA](/wiki/llama) and Llama branding since 2023.[1] [Llama 3.3](/wiki/llama_3_3) and the broader Llama 3 generation introduced strong text-only performance, but those models relied on separate vision adapters added after pretraining rather than native multimodal training.

For Llama 4, Meta pursued a different approach. The company trained a family of models from scratch with multimodal data included in pretraining, so that text and image understanding are integrated at the architectural level rather than grafted on afterward.[1] Meta also adopted a Mixture of Experts architecture for the first time in the Llama series, following similar moves by other labs including [Google DeepMind](/wiki/google_deepmind) with Gemini and [Mistral AI](/wiki/mistral_ai) with Mixtral.

The Llama 4 family was announced under the tagline "the beginning of a new era of natively multimodal AI innovation."[1] At launch, Scout and Maverick weights were made available for download from llama.com (subject to license acceptance), and the models were immediately integrated into Meta's own Meta.ai assistant product as the underlying backbone.[1]

## What is Llama 4 Scout?

### Specifications

Llama 4 Scout is a sparse MoE model with 16 experts and 17 billion active parameters per forward pass.[1] The full parameter count across all experts is approximately 109 billion. During inference, only the active 17 billion parameters are used for any given token, making the per-token compute comparable to a dense 17B model despite the much larger total capacity.[1]

Scout was pretrained on approximately 40 trillion tokens of multimodal data, more than Maverick's 22 trillion tokens, which Meta attributes to Scout's training being more data-efficient.[2] The pretraining context length was 256,000 tokens; the instruct-tuned version extends this to 10 million tokens through a mid-training phase on long-context data.[2]

With Int4 quantization, Scout fits on a single NVIDIA H100 GPU, making it the more accessible of the two released models for self-hosted deployments.[1]

### How big is Scout's context window?

The 10-million token context window is one of Scout's most distinctive features. Meta calls it an "industry-leading context window of 10M," and at release it was the longest context window of any publicly available open-weight model.[1] To put the scale in perspective, 10 million tokens can accommodate roughly 7,500 pages of text, around 20 hours of transcribed speech, or large multi-file codebases in a single prompt.[1]

Meta reports near-perfect retrieval accuracy on Needle-in-a-Haystack evaluations up to the full 10-million token limit.[1] In the multilingual translation benchmark MTOB (half-book), Scout scored 42.2 on English-to-Kinyarwanda and 36.6 on Kinyarwanda-to-English tasks.[1]

The extended context is enabled by the iRoPE architecture (described in its own section below), which combines layers without positional encoding, chunked attention on most layers, and inference-time temperature scaling.[2]

### Benchmark performance

On standard academic benchmarks, Scout scores 79.6 on MMLU, 74.3 on MMLU Pro, 57.2 on GPQA Diamond, 50.3 on MATH, 32.8 on LiveCodeBench, 88.8 on ChartQA, and 94.4 on DocVQA.[1] On the multilingual math benchmark MGSM, Scout achieves 90.6.[1]

Meta positions Scout as outperforming Gemma 3, Gemini 2.0 Flash-Lite, and Mistral 3.1 across a range of benchmarks at comparable or lower inference cost.[1]

### What is Scout used for?

Meta designed Scout for applications that require processing very long documents or large codebases in a single pass. Specific scenarios include enterprise document retrieval (searching across thousands of files to answer a query), multi-document summarization, reasoning over lengthy technical manuals, and analyzing large software repositories.[1] The single-GPU inference footprint also makes Scout well-suited for organizations that want to run a capable open model without multi-GPU server infrastructure.

## What is Llama 4 Maverick?

### Specifications

Llama 4 Maverick has 128 experts and 17 billion active parameters per token, with approximately 400 billion total parameters across all experts.[1] The instruct-tuned version supports a 1-million token context window. Maverick's architecture alternates between dense layers and MoE layers, unlike Scout which uses MoE throughout.[2]

Maverick was pretrained on approximately 22 trillion tokens.[2] It was also codistilled from Llama 4 Behemoth (see the distillation section below), which Meta credits as a significant driver of quality gains over what pretraining alone would have produced.[1]

Maverick requires a single NVIDIA H100 DGX host (8 GPUs) for inference in BF16 precision. FP8 weights are also available for reduced memory footprint.[2]

### Benchmark performance

Maverick scores 85.5 on MMLU, 80.5 on MMLU Pro, 69.8 on GPQA Diamond, 61.2 on MATH, 43.4 on LiveCodeBench, 90.0 on ChartQA, and 94.4 on DocVQA.[1] On MGSM, Maverick scores 92.3. On Multilingual MMLU, it scores 84.6.[1]

On the MMMU image reasoning benchmark, Maverick scores 73.4 and Scout scores 69.4. On MathVista, Maverick scores 73.7 and Scout scores 70.7.[1]

Compared to other models at similar active parameter counts, Maverick's GPQA Diamond score of 69.8 is notably higher than GPT-4o's reported score of 53.6 on the same benchmark, though direct comparisons across benchmark versions and evaluation conditions require caution.[1] Meta also claims Maverick achieves "comparable results to the new DeepSeek v3 on reasoning and coding, at less than half the active parameters."[1]

### How did Maverick rank on LMArena?

At launch, Meta highlighted that an experimental chat version of Maverick achieved an ELO score of 1417 on the LMArena chatbot leaderboard, placing it second overall, just behind Google's Gemini 2.5 Pro.[1] Meta wrote that Maverick "offers a best-in-class performance to cost ratio with an experimental chat version scoring ELO of 1417 on LMArena."[1] This claim later became controversial (see the LMArena controversy section below).[5][6]

### What is Maverick used for?

Maverick targets applications requiring strong multimodal reasoning alongside text generation quality. Described use cases include customer support systems that process user-uploaded images, multilingual assistants (Llama 4 models support 200 languages, with 100 or more having over one billion training tokens each), enterprise question-answering over rich media, and creative generation tasks where vision and language understanding combine.[1] Meta describes Maverick as offering the best performance-to-cost ratio in its class among models available at the time of release.[1]

## What is Llama 4 Behemoth?

Llama 4 Behemoth is the largest model in the Llama 4 family, with approximately 288 billion active parameters and 16 experts across a total of nearly two trillion parameters.[1] At the time of the April 2025 announcement, Behemoth was still in training; Meta disclosed it as a preview to describe the broader Llama 4 research direction and its role as a teacher model.[1]

Meta states that Behemoth, even in its incomplete training state, "outperforms GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on several STEM benchmarks," including MATH-500 and GPQA Diamond.[1] The full model is intended for agentic and reasoning tasks that require the largest available capacity.

Behemoth's primary role in the Llama 4 release is as the teacher in the codistillation process used to train Maverick (and to a lesser extent Scout).[1] Unlike Scout and Maverick, Behemoth weights are not publicly available, and Meta has not committed to an open-weight release.

Behemoth was pretrained on more than 30 trillion tokens using FP8 precision across 32,000 GPUs, achieving 390 teraFLOPS per GPU during training.[2] Meta used the MetaP method to set hyperparameters, which allowed stable transfer of learning rate and batch size schedules across model scales.[1]

## How does the iRoPE architecture enable long context?

All Llama 4 models use what Meta calls the iRoPE architecture for positional encoding and long-context support.[2] The name combines "i" (for interleaved or, by Meta's framing, a nod toward infinite context) with "RoPE" ([Rotary Position Embeddings](/wiki/rotary_position_embedding)), the positional encoding scheme used in Llama 3 and many other recent transformer models.[2]

iRoPE works by interleaving two different types of attention layers throughout the model:

The majority of layers (three out of every four) use standard RoPE positional encodings with chunked attention. In chunked attention, the input sequence is divided into fixed chunks (8,192 tokens per chunk for Llama 4), and each token only attends within its chunk plus to any global context tokens. This keeps the computational cost manageable even for very long inputs.[2]

Every fourth layer is a NoPE layer (No Positional Encoding). NoPE layers apply full causal attention across the entire input sequence without any positional information encoded. Because these layers have no position bias, they can generalize to sequence lengths far beyond those seen in training, acting as global context aggregators that can relate tokens from the beginning and end of a very long document.[2]

At inference time, Meta applies temperature scaling on the attention scores in the NoPE layers. This prevents attention entropy collapse in long sequences, where without scaling, attention scores tend to become uniform and lose discriminative power as the sequence grows longer. Temperature scaling was applied post-training without requiring retraining.[2]

Scout additionally uses RMS normalization of query and key vectors in the RoPE layers (QK normalization), which provides further training stability.[2]

This combination of chunked RoPE attention, NoPE full-attention layers, and temperature scaling is what allows Scout to generalize to 10 million tokens from a pretraining context of 256,000 tokens.[2]

## How does native multimodality work in Llama 4?

Previous Llama models were text-only at the base and added visual understanding through separate adapter modules trained after the language backbone was complete. This late-fusion approach, also called post-hoc multimodality, typically limits how deeply visual and textual representations are integrated.[1]

Llama 4 uses early fusion instead. Text tokens and image tokens are fed into the same transformer backbone from the beginning, processed jointly through all layers.[1] This means the model develops shared representations for visual and linguistic content rather than treating them as separate modalities that communicate through a narrow adapter interface.

The vision input is processed by a MetaCLIP-based vision encoder. Meta adapted the encoder specifically to work with the Llama backbone, training the encoder in a frozen-Llama configuration to align visual representations with the language model's internal representation space.[1] The encoder converts images into a sequence of visual tokens that are interleaved with text tokens before entering the main transformer.

Joint pretraining on text, images, and video data allows the model to learn natural correspondences between modalities at scale. Meta trained on diverse image-text pairs, documents with embedded figures, and video with associated transcripts.[1] Supporting up to multiple images in a single prompt, Llama 4 models can also anchor responses to specific regions within an image, a capability Meta calls being "best-in-class on image grounding, able to align user prompts with relevant visual concepts."[1]

Native multimodality is one of the main reasons Meta cites for the strong performance on document understanding benchmarks like DocVQA (94.4 for both Scout and Maverick) and chart comprehension benchmarks like ChartQA (88.8 for Scout, 90.0 for Maverick).[1]

## How were Scout and Maverick post-trained?

After pretraining, Scout and Maverick went through a multi-stage post-training process. The pipeline consisted of lightweight supervised fine-tuning (SFT), followed by online reinforcement learning (RL), and then a final lightweight direct preference optimization (DPO) phase.[1]

For Maverick, Meta removed more than 50 percent of data tagged as easy from the SFT stage, concentrating the fine-tuning on harder examples where the model needed more guidance.[1] The online RL phase used adaptive difficulty filtering: prompts where the model had no meaningful gradient signal (because it already answered correctly with high confidence, yielding zero advantage estimates) were dropped from training batches. This continuous filtering kept the RL training focused on the edge of the model's current ability.[1]

For Behemoth, the post-training process was even more aggressive. Meta pruned 95 percent of SFT data and relied heavily on large-scale RL with hard prompt sampling.[1] The goal for Behemoth was to push STEM reasoning performance as far as possible without the distillation constraints that apply to the smaller models.

Bias and over-refusal were also targets of the post-training work. Meta reduced political and social topic refusal rates from around 7 percent in Llama 3.3 down to below 2 percent in Llama 4.[1] Unequal response rates across demographic groups were reduced to below 1 percent.[1]

## How was Maverick distilled from Behemoth?

Llama 4 Maverick was trained using a process Meta calls codistillation from Llama 4 Behemoth. Standard [knowledge distillation](/wiki/knowledge_distillation) trains a smaller student model to mimic the output distributions of a larger teacher model, typically after the teacher is fully trained. Codistillation runs the student and teacher training processes concurrently, which reduces cost by amortizing the teacher's forward passes over the student's full training run.[1]

For the majority of training tokens, Behemoth's output distribution (the soft targets) was already available from the teacher's own training run, so the student could use those targets without requiring additional teacher inference. For any new data added specifically to the student's training mixture, Meta ran Behemoth forward passes to generate fresh distillation targets.[1]

Meta developed a novel dynamic distillation loss function that adjusts the weighting between soft targets (Behemoth's output distribution) and hard targets (ground truth labels) over the course of training.[1] Early in training, when the student model is far from convergence, hard targets provide stronger gradient signal. As training progresses and the student stabilizes, soft targets become more informative because they encode the teacher's nuanced uncertainty. The dynamic weighting captures this shift automatically.

Meta reports that codistillation produced substantial quality improvements in Maverick compared to training without a teacher, particularly on reasoning and knowledge-intensive tasks.[1]

## Is Llama 4 open source, and what does the license allow?

Llama 4 Scout and Maverick are released under the Llama 4 Community License Agreement.[4] The license is best described as open-weight rather than open-source in the strict OSI sense, because it carries usage restrictions. It permits commercial use for most organizations, with two significant restrictions:

Organizations with more than 700 million monthly active users as of April 2025 must obtain a separate license from Meta, granted at Meta's sole discretion.[4] This clause is aimed at hyperscale technology platforms that could use Llama to build competing products at scale. The 700 million MAU threshold applies to the entire legal entity, including subsidiaries and affiliates.

The multimodal capabilities (vision) of Llama 4 models cannot be used by organizations or individuals domiciled in the European Union, or by companies whose principal place of business is in the EU.[4] The text-only paths are not restricted. Meta did not publicly explain the specific regulatory reason for this restriction, but it coincides with obligations under the EU AI Act and related data protection frameworks. This restriction applies to the licensee, not to end users of a product built with the models.

Derivative models and products must include "Built with Llama" attribution.[4] The license does not permit using Llama 4 outputs to train other foundational AI models that compete with Meta's products.

## How do Scout and Maverick perform on benchmarks?

The table below shows performance on selected benchmarks for Llama 4 Scout and Llama 4 Maverick, compared to [GPT-4o](/wiki/gpt_4o), [DeepSeek V3](/wiki/deepseek_v3), and Gemini 2.0 Flash.[1]

| Benchmark | Category | Scout | Maverick | GPT-4o | DeepSeek V3 | Gemini 2.0 Flash |
|---|---|---|---|---|---|---|
| MMLU | Knowledge | 79.6 | 85.5 | 85.7 | 88.5 | 85.2 |
| MMLU Pro | Knowledge | 74.3 | 80.5 | 72.6 | 75.9 | 77.6 |
| GPQA Diamond | Science reasoning | 57.2 | 69.8 | 53.6 | 59.4 | 60.1 |
| MATH | Mathematics | 50.3 | 61.2 | 74.6 | 87.1 | 89.7 |
| LiveCodeBench | Coding | 32.8 | 43.4 | 32.3 | 49.2 | 44.5 |
| MMMU | Image reasoning | 69.4 | 73.4 | 69.1 | N/A | 70.7 |
| MathVista | Visual math | 70.7 | 73.7 | 63.8 | N/A | 73.1 |
| ChartQA | Chart understanding | 88.8 | 90.0 | 85.7 | N/A | 87.2 |
| DocVQA | Document understanding | 94.4 | 94.4 | 91.1 | N/A | 92.1 |
| MGSM | Multilingual math | 90.6 | 92.3 | 90.5 | 91.1 | 89.2 |

Note: Benchmark scores are drawn from Meta's official model card and the Hugging Face release post.[1][2] Cross-benchmark comparisons require caution because evaluation conditions, prompt formats, and model versions vary across organizations.

## What was the LMArena controversy?

When Meta announced Llama 4 on April 5, 2025, it cited an ELO score of 1417 on the LMArena chatbot leaderboard as evidence of Maverick's quality.[1] LMArena is a widely followed benchmark where human evaluators compare AI model outputs in blind side-by-side tests.

The model Meta submitted to LMArena was not the same as the model released to the public. The submitted version was labeled "Llama-4-Maverick-03-26-Experimental," a variant tuned specifically for human preference rather than general instruction-following.[5] The experimental version consistently produced verbose, heavily emoji-laden responses, a style that tends to score well on preference-based evaluations but that many users find inappropriate for professional or technical applications.[5]

When the publicly released Llama-4-Maverick-17B-128E-Instruct was separately evaluated on LMArena, it ranked approximately 32nd on the leaderboard, well below the experimental version's second-place finish and below established competitors including GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro, all of which were older models.[6]

Meta acknowledged the distinction without apologizing, stating that "Llama-4-Maverick-03-26-Experimental is a chat optimized version we experimented with that also performs well on LMArena. We have now released our open source version and will see how developers customize Llama 4 for their own use cases."[5] LMArena updated its leaderboard policies in response, stating that "Meta's interpretation of our policy did not match what we expect from model providers," and committing to clearer standards for which model variants can be submitted.[5] The platform also published the more than 2,000 battles from the experimental submission so researchers could examine the preference data.[5]

The episode drew wider commentary about the reliability of preference-based benchmarks.[7] Because LMArena rewards conversational style, verbosity, and formatting choices that appeal to casual evaluators, models can be optimized to perform well on the benchmark without those optimizations translating to better real-world performance across technical or professional tasks.[7]

## How does Llama 4 compare to DeepSeek V3, GPT-4o, and Gemini 2.0 Flash?

The table below focuses on the positioning of Maverick, the stronger of the two released models, against the three main competitors Meta cited at launch.[1]

| Model | Total params | Active params | Context | Multimodal | Open weight |
|---|---|---|---|---|---|
| [Llama 4 Scout](/wiki/llama_4_scout_maverick) | 109B | 17B | 10M | Yes | Yes |
| [Llama 4 Maverick](/wiki/llama_4_scout_maverick) | 400B | 17B | 1M | Yes | Yes |
| [GPT-4o](/wiki/gpt_4o) | Not disclosed | Not disclosed | 128K | Yes | No |
| [DeepSeek V3](/wiki/deepseek_v3) | 671B | 37B | 128K | No | Yes |
| Gemini 2.0 Flash | Not disclosed | Not disclosed | 1M | Yes | No |

On reasoning benchmarks, Maverick's GPQA Diamond score (69.8) is notably stronger than GPT-4o (53.6) but weaker than DeepSeek V3 on MATH (87.1 vs. 61.2).[1] Maverick shows competitive results on coding benchmarks (LiveCodeBench 43.4) that are broadly comparable to Gemini 2.0 Flash and close to DeepSeek V3, despite having roughly half the active parameters.[1]

For document and chart understanding tasks where multimodal capabilities matter, Maverick and Scout are competitive with or ahead of GPT-4o, which is notable because document AI has historically been a strength of closed models with native vision integration.[1]

DeepSeek V3, which at the time of Llama 4's release was one of the strongest open-weight text models, does not natively process images, making direct multimodal comparison impossible for vision tasks.

The clearest practical difference is in context length. Scout's 10-million token window is substantially larger than any competitor in the table.[1] Gemini 2.0 Flash also supports up to 1 million tokens, matching Maverick, while GPT-4o and DeepSeek V3 are limited to 128,000 tokens.

## What are Scout and Maverick used for?

Scout is aimed at applications that require long-context reasoning within a resource-efficient deployment footprint. Practical scenarios include:

Enterprise knowledge retrieval: processing entire document libraries, SharePoint repositories, or internal knowledge bases in a single context window, without the chunking and retrieval pipeline complexity required by shorter-context models.

Code analysis: loading complete multi-file codebases for security auditing, refactoring analysis, or automated code review without losing cross-file context.

Long document processing: summarizing legal contracts, technical specifications, financial filings, or multi-volume research reports in a single pass.

Maverick targets applications where multimodal reasoning quality is the primary requirement alongside text generation:

Document AI: extracting structured data from invoices, forms, charts, and reports where text and visual layout are both relevant.

Customer support: handling support tickets that include user-uploaded screenshots, product photos, or error message images alongside text descriptions.

Content analysis: processing mixed media inputs for moderation, classification, or information extraction at scale.

Multilingual applications: Meta's training on 200 languages with 1 billion or more tokens each for over 100 of them makes both models notably stronger on non-English languages than most comparably sized models.[1]

## Where can you run Llama 4 Scout and Maverick?

Both models were available on the day of announcement through multiple channels. Model weights can be downloaded from llama.com after accepting the license terms.[1] Hugging Face hosts the models at meta-llama/Llama-4-Scout-17B-16E and meta-llama/Llama-4-Maverick-17B-128E-Instruct, with support for the Transformers library starting from version 4.51.0.[2]

Cloud providers moved quickly to offer hosted access. Amazon Web Services made both models available in Amazon Bedrock as a serverless option in the US East and US West regions on the day of release, with cross-region inference available in US East (Ohio).[12] Microsoft Azure offered the models through Azure AI Foundry and Azure Databricks. Google Cloud also announced same-day support.[1]

Third-party inference providers including Groq, Together.ai, and Fireworks also offered API access at launch. Groq reported output speeds of over 400 tokens per second for Scout, enabled by its LPU (Language Processing Unit) hardware. IBM made both models available in its watsonx.ai platform.

For local deployment, Scout can run on a single H100 GPU with Int4 quantization using tools like llama.cpp or Ollama.[1] Maverick requires a multi-GPU setup, with 8 GPUs recommended for BF16 inference using tensor parallelism.[2]

## How was Llama 4 received?

The release of Scout and Maverick was broadly covered as a significant step in Meta's open-weight model program and as a direct competitive response to [DeepSeek](/wiki/deepseek), which had attracted widespread attention in early 2025 by demonstrating that high-quality models could be trained at lower cost than previously assumed.[8]

Positive reception focused on the multimodal integration quality (particularly document and chart understanding scores that matched or exceeded closed models), the long context windows, and the inference efficiency enabled by the MoE architecture. The fact that both models are open-weight drew particular interest from enterprises and researchers who want to fine-tune or self-host models without per-token API costs.[8]

The codistillation from Behemoth was noted as an interesting training-time technique, though some observers pointed out that the teacher model being used to improve the student is itself not publicly available, which limits independent replication.

Criticism centered primarily on the LMArena benchmark controversy (see above), which led to questions about the reliability of the performance claims in Meta's launch materials.[7] Security researchers at Virtue AI and ProtectAI published redteaming analyses shortly after release, finding that both Scout and Maverick had jailbreak vulnerability rates in the medium-risk range (52 to 58 percent success rates on standard adversarial test suites), with Maverick showing better compliance behavior than Scout.[10][11]

Some practitioners noted that Scout's real-world coding performance felt weaker than benchmarks suggested, with the model occasionally struggling on complex multi-step programming tasks where [DeepSeek V3](/wiki/deepseek_v3) or GPT-4o performed more reliably.

## What are the limitations of Llama 4 Scout and Maverick?

Several limitations were documented at and after release:

Hallucination rate: Both models exhibit factual hallucination in multi-step reasoning chains. Maverick's rate is lower than Scout's but higher than GPT-4.5 based on third-party evaluations. Neither model should be used for high-stakes factual retrieval without external grounding.

Coding reliability: Despite competitive LiveCodeBench scores, developer feedback in the weeks after release was mixed. Scout in particular drew criticism for difficulty with complex algorithmic tasks and for producing plausible-but-incorrect code at a higher rate than DeepSeek V3.

EU restriction: The multimodal license restriction means that EU-based organizations cannot legally build or deploy products that use Llama 4's vision capabilities, substantially narrowing the open-weight advantage for European teams.[4]

Behemoth availability: The teacher model used to improve Maverick through codistillation is not publicly available and was not fully trained at announcement. This limits both independent evaluation of the distillation claims and any attempt to reproduce the training process.[1]

Context quality at scale: While Meta reports near-perfect needle-in-a-haystack accuracy, sustained reasoning over very long contexts (millions of tokens) introduces real-world latency and memory constraints that make the 10-million token window practically accessible only on hardware with substantial RAM and fast interconnect.[1]

## ELI5: Llama 4 Scout and Maverick in plain English

Imagine a big team of specialists instead of one know-it-all. When you ask a question, only the two or three specialists who are good at that topic step forward to answer, while everyone else rests. That is the Mixture of Experts idea: the model is huge (lots of specialists), but only a small part of it works on any single word, so it stays fast and cheap to run. Scout is the smaller team that can read a truly enormous amount at once (think thousands of pages in one go) and still fit on one powerful computer chip. Maverick is the bigger team that is especially good at understanding pictures and charts along with text. The catch is that when Meta showed off Maverick on a popularity contest leaderboard, it quietly used a special show-off version that wrote chatty, emoji-filled answers to win votes, and the version everyone could actually download did not do nearly as well.

## See also

- [Llama 4](/wiki/llama_4)
- [LLaMA](/wiki/llama)
- [Llama 3.3](/wiki/llama_3_3)
- [Mixture of Experts](/wiki/mixture_of_experts)
- [Multimodal AI](/wiki/multimodal_ai)
- [DeepSeek V3](/wiki/deepseek_v3)
- [GPT-4o](/wiki/gpt_4o)
- [Knowledge distillation](/wiki/knowledge_distillation)
- [Rotary position embedding](/wiki/rotary_position_embedding)

## References

1. Meta AI. "The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation." ai.meta.com/blog, April 5, 2025. https://ai.meta.com/blog/llama-4-multimodal-intelligence/
2. Hugging Face. "Welcome Llama 4 Maverick & Scout on Hugging Face." huggingface.co/blog, April 5, 2025. https://huggingface.co/blog/llama4-release
3. Meta AI. "Llama 4 Scout - Hugging Face Model Card." meta-llama/Llama-4-Scout-17B-16E. https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E
4. Meta AI. "Llama 4 Community License Agreement." llama.com, 2025. https://www.llama.com/llama4/license/
5. The Register. "Meta accused of Llama 4 bait-n-switch to juice LMArena rank." theregister.com, April 8, 2025. https://www.theregister.com/2025/04/08/meta_llama4_cheating/
6. TechCrunch. "Meta's vanilla Maverick AI model ranks below rivals on a popular chat benchmark." techcrunch.com, April 11, 2025. https://techcrunch.com/2025/04/11/metas-vanilla-maverick-ai-model-ranks-below-rivals-on-a-popular-chat-benchmark/
7. TechCrunch. "Meta's benchmarks for its new AI models are a bit misleading." techcrunch.com, April 6, 2025. https://techcrunch.com/2025/04/06/metas-benchmarks-for-its-new-ai-models-are-a-bit-misleading/
8. VentureBeat. "Meta's answer to DeepSeek is here: Llama 4 launches with long context Scout and Maverick models, and 2T parameter Behemoth on the way!" venturebeat.com, April 5, 2025. https://venturebeat.com/ai/metas-answer-to-deepseek-is-here-llama-4-launches-with-long-context-scout-and-maverick-models-and-2t-parameter-behemoth-on-the-way
9. Meta AI. "Llama 4 model page." llama.com/models/llama-4/, 2025. https://www.llama.com/models/llama-4/
10. Virtue AI. "Llama 4 Scout & Maverick Redteaming Analysis." blog.virtueai.com, April 8, 2025. https://blog.virtueai.com/2025/04/08/llama-4-scout-maverick-redteaming-analysis/
11. Protect AI. "Llama 4 Series Vulnerability Assessment: Scout vs. Maverick." protectai.com, 2025. https://protectai.com/blog/vulnerability-assessment-llama-4
12. AWS. "Llama 4 models from Meta now available in Amazon Bedrock serverless." aws.amazon.com, April 2025. https://aws.amazon.com/blogs/aws/llama-4-models-from-meta-now-available-in-amazon-bedrock-serverless/