Llama 2
Last reviewed
Sources
No citations yet
Review status
Needs citations
Revision
v7 ยท 8,110 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Sources
No citations yet
Review status
Needs citations
Revision
v7 ยท 8,110 words
Add missing citations, update stale details, or suggest a clearer explanation.
Llama 2 (Large Language Model Meta AI 2; sometimes stylized LLaMA 2 after the original paper's backronym) is a family of large language models developed and released by Meta on July 18, 2023 [1][2]. As the second generation of Meta's Llama series, it represented a turning point in the open-weight AI movement by making powerful foundation models freely available for both research and commercial use under a single bespoke license. The release included pretrained base models and chat-tuned variants at three publicly distributed sizes (7 billion, 13 billion, and 70 billion parameters), all trained on 2 trillion tokens of publicly available data with a 4,096-token context window [1]. Llama 2 was accompanied by an unusually detailed 76-page research paper, "Llama 2: Open Foundation and Fine-Tuned Chat Models" (arXiv:2307.09288) [1], and a new community license that allowed commercial deployment subject to a small number of conditions. The release was made jointly with Microsoft as Meta's "preferred partner," positioning Llama 2 as the centerpiece of a broader open-vs-closed strategic realignment in the generative AI industry [2][3].
Llama 2 quickly became the default foundation model for thousands of academic papers, startup products, and community fine-tunes throughout late 2023 and the first half of 2024, before being superseded by Llama 3 in April 2024 [4]. Even after newer Meta models arrived, Llama 2 has remained a common teaching example because its architecture is well documented, its weights are freely downloadable, and its size range covers everything from a laptop-friendly 7B to a research-cluster 70B. Its release is widely regarded as the catalyst for the modern open-weight LLM ecosystem [4][5].
Llama 2 is Meta's second-generation family of open-weight large language models, released on July 18, 2023 in 7B, 13B, and 70B parameter sizes, each as a pretrained base model and a chat-tuned variant (Llama 2-Chat), all trained on 2 trillion tokens with a 4,096-token context window and distributed for free commercial use under the Llama 2 Community License [1][2][23]. Its defining contribution was permission: where the original Llama (February 2023) was research-only, Llama 2 was the first high-quality model family a company could legally ship in a commercial product without paying a license fee, subject only to a handful of conditions (most notably a clause covering services with more than 700 million monthly active users) [23][24].
In the paper's own framing, the authors state: "In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases." [1] They further report that these models "outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models." [1]
Llama 2 was announced and released on July 18, 2023, jointly by Meta and Microsoft at Microsoft's Inspire partner conference [2][3]. The accompanying paper (arXiv:2307.09288) was posted to arXiv the same day [1]. It followed the original Llama (Llama 1) by about five months (February 24, 2023) and was itself superseded by Llama 3 on April 18, 2024 [6][34].
Llama 2 is best described as open weight rather than open source. The weights and the Llama 2 Community License are public and permit free commercial use, but the license is not an Open Source Initiative approved license: it imposes an acceptable use policy and a 700 million monthly active user threshold, and Meta did not release the training data [23][26]. The Open Source Initiative stated in 2023 that "Meta's LLaMa 2 license is not Open Source." [26] The fuller debate is covered under the open source debate below.
Llama 2 is used as a general-purpose text generation and dialogue model and, more importantly, as a base for fine-tuning. The base models support downstream task adaptation, while the Llama 2-Chat variants are tuned for multi-turn assistant conversations [1]. Because the weights are freely downloadable and run on commodity hardware (the 7B chat model fits on a single consumer GPU and runs on Apple silicon via llama.cpp GGUF quantization), Llama 2 became the most fine-tuned foundation model of 2023, seeding tens of thousands of community derivatives for chat, coding, medical question answering, vision-language tasks, and safety classification [4][13].
Meta released the original Llama (often retroactively called Llama 1) on February 24, 2023 [6]. The first-generation models were distributed under a non-commercial research-only license, with weights gated behind an application form on Meta's website. The weights for all four sizes (7B, 13B, 33B, and 65B) were leaked on 4chan within a week of the announcement and reposted across BitTorrent and Hugging Face mirrors, making them effectively public in practice even though the license restricted commercial use [7].
The community activity that followed demonstrated demand that closed APIs could not satisfy. Within six weeks of the leak, Stanford released Alpaca, Llama 1 7B fine-tuned on 52,000 instruction-following demonstrations generated using OpenAI's text-davinci-003, at a marginal compute cost reported as under $600 [8]. A team from LMSYS, UC Berkeley, Stanford, and UCSD followed with Vicuna, fine-tuned on roughly 70,000 ShareGPT user-shared ChatGPT conversations and announced as reaching "more than 90% of ChatGPT and Bard quality" in their own evaluation [9]. Other notable Llama 1 derivatives included Koala (Berkeley), GPT4All (Nomic), Guanaco, and Open Assistant's first model checkpoints. All operated in legal grey territory because they were built on weights that had been leaked rather than properly licensed for commercial use.
By mid-2023 the situation was clear: there was overwhelming developer interest in running and fine-tuning large language models on their own hardware, but the supply of legally usable open weights was limited to older or smaller models (Pythia, GPT-J, RedPajama, MPT, Falcon) that lagged behind the closed frontier in quality. Meta said it had received more than 100,000 access requests for Llama 1, an indicator of latent demand that Meta cited as one motivation for an open commercial release [2].
The timing was also strategically significant. In mid-2023 OpenAI's GPT-4 and ChatGPT dominated public attention, and the prevailing narrative in the industry favored closed, proprietary models accessed via API. OpenAI's switch from a research-publishing nonprofit to a closed-model commercial product had alienated portions of the academic community and prompted criticism from researchers including Yann LeCun, who served as Meta's chief AI scientist [10]. Mark Zuckerberg framed Meta's open strategy in subsequent earnings calls and interviews as a way to commoditize the input layer that Meta's competitors profited from selling, a "complementary good" argument that compared open foundation models to Linux as a foundation for paid services [11]. The Llama 2 release was the most direct expression of this strategy.
Meta and Microsoft jointly announced Llama 2 on July 18, 2023, at Microsoft's annual Inspire partner conference in Las Vegas [2][3]. Microsoft chief executive Satya Nadella publicly endorsed the partnership on stage. The companies' joint statement designated Microsoft as "the preferred partner for Llama 2" and emphasized commercial availability on Microsoft Azure and Windows in addition to direct download from Meta and from Hugging Face [2][3]. Meta's own announcement stated plainly that "Llama 2 is free for research and commercial use." [2] Within hours of the announcement, the model weights were available as direct downloads via a request form on Meta's website (typically approved within a day) and as hosted artifacts on the meta-llama Hugging Face organization [12]. The 7B and 13B GGML and GPTQ quantizations followed within 24 hours through community contributors, and llama.cpp added Llama 2 support the same week [13].
The release date was chosen to align with Inspire, but the broader context also mattered: it came two weeks before Meta's Q2 2023 earnings release, in a period of intense competition with Google's recently-announced PaLM 2 (May 2023) and OpenAI's GPT-4 (March 2023). Llama 2 was Meta's first major model release after the leak of Llama 1 forced the company to confront how to handle openness deliberately rather than reactively.
The accompanying paper, "Llama 2: Open Foundation and Fine-Tuned Chat Models" (arXiv:2307.09288), was published on July 18, 2023, the same day as the model release. The lead authors were Hugo Touvron, Louis Martin, Kevin Stone, and Peter Albert, with more than 50 additional contributors from Meta's GenAI organization [1]. Touvron had also been the lead author of the original Llama paper, providing continuity between the two model families. The paper runs to 76 pages and reads more like an engineering tech report than a conventional academic submission, with extensive appendices on data filtering, annotation guidelines, safety taxonomy, and example outputs. Several Meta researchers reported on social media that the level of disclosure was deliberately calibrated to be reproducible by other research groups working at smaller scale, even if the training data could not be shared.
Meta uses the spelling "Llama 2" with title-case in its own documentation, but the original paper title and the academic literature continue to use "LLaMA 2" as a backronym for "Large Language Model Meta AI 2." Both spellings appear in primary sources and are interchangeable in practice. Starting with Llama 3, Meta dropped the all-caps form entirely, and "Llama" is now the canonical product name [14].
Llama 2 was released in three parameter sizes, each available as both a pretrained base model and a fine-tuned chat model optimized for dialogue. A 34B parameter variant was trained but withheld from public release; Meta wrote in the paper that "due to a lack of time to sufficiently red team, we are postponing the release of our 34B model" [1]. As of 2026 the 34B base text model has never been released, although a 34B Code Llama variant derived from the same architecture was released the following month [15].
| Model | Parameters | Layers | Heads | KV Heads | Hidden Dim | Context | Attention | Public Release |
|---|---|---|---|---|---|---|---|---|
| Llama 2 7B | 7B | 32 | 32 | 32 | 4,096 | 4,096 | MHA | Base + Chat |
| Llama 2 13B | 13B | 40 | 40 | 40 | 5,120 | 4,096 | MHA | Base + Chat |
| Llama 2 34B | 34B | 48 | 56 | 8 | 7,168 | 4,096 | GQA | Withheld |
| Llama 2 70B | 70B | 80 | 64 | 8 | 8,192 | 4,096 | GQA | Base + Chat |
The base models were designed for general-purpose text generation and could be fine-tuned for specific downstream tasks. The chat variants (Llama 2-Chat) were tuned for multi-turn dialogue through supervised fine-tuning and reinforcement learning from human feedback (RLHF) [1].
The three publicly released sizes shared the same transformer decoder-only backbone, with the principal difference that the 70B (and unreleased 34B) used grouped-query attention rather than standard multi-head attention [1]. This is an important detail that is often misreported in secondary sources: GQA in Llama 2 applies only to the 34B and 70B variants. The 7B and 13B used full multi-head attention with one key-value pair per query head [1][12]. GQA shares each set of key and value projections across multiple query heads, reducing memory bandwidth requirements during inference and improving throughput at the largest scale.
Llama 2 retained the core architectural choices of Llama 1 while making targeted improvements aimed at long-context performance and inference efficiency. The model is a standard decoder-only transformer with several specific design choices that have since become common in open-weight LLMs.
Following GPT-3 and Llama 1, the model applies RMSNorm (Root Mean Square Layer Normalization, Zhang and Sennrich 2019) before each transformer sub-layer rather than after [1][16]. Pre-normalization improves training stability, particularly at scale, by keeping the residual stream's variance bounded as gradients flow back through deep networks. RMSNorm itself is a simplified variant of LayerNorm that drops the mean-centering step and the learnable bias, reducing parameter count and compute per layer while empirically matching full LayerNorm's quality.
The feed-forward network uses the SwiGLU activation introduced by Noam Shazeer in 2020 [17]. SwiGLU combines a gated linear unit with the Swish (also known as SiLU) activation, multiplying two linear projections of the input where one is passed through Swish. Empirically this provides better performance than standard ReLU or GELU activations at the same parameter budget, with a small constant cost from the extra projection. To keep the parameter count of the FFN comparable to a standard GELU MLP, the Llama 2 design uses an intermediate dimension of approximately 8/3 times the hidden dimension instead of the conventional 4x.
Instead of absolute or learned positional encodings, Llama 2 uses rotary position embeddings (RoPE), introduced by Su et al. (2021) [18]. RoPE encodes position by rotating the query and key vectors at each layer through angles that depend on token position, so that the dot product between two rotated vectors naturally encodes their relative position. RoPE generalizes more gracefully to sequence lengths not seen during training and forms the basis for later context-extension techniques like NTK-aware and YaRN scaling.
The 70B model (and the unreleased 34B) uses grouped-query attention (GQA, Ainslie et al. 2023) with 8 key-value heads shared across the query heads [19]. For the 70B that means 8 KV heads paired with 64 query heads, an 8x reduction in the size of the key-value cache compared to full multi-head attention. The smaller cache makes long-sequence inference dramatically more memory-efficient, which becomes critical when serving the 70B model at production scale. GQA sits between standard multi-head attention (one KV pair per query head) and multi-query attention (a single KV pair shared across all query heads), trading a small quality loss for a large efficiency gain. The 7B and 13B models, which fit more easily in GPU memory, retain conventional multi-head attention with one KV pair per query head [1].
All Llama 2 models support a context window of 4,096 tokens, double the 2,048-token context of Llama 1 [1]. The longer window allows the model to process roughly 3,000 words of input plus output and was a key enabler for the chat use case, where multi-turn conversations and long system prompts can quickly exhaust shorter contexts. Even at 4K, however, Llama 2 was considerably shorter than Anthropic's Claude 100K-token context already available at the time of release, a gap Meta acknowledged in the paper [1].
The vocabulary is 32,000 tokens trained with byte-pair encoding (BPE) using the SentencePiece implementation, identical to Llama 1 [1]. Numbers are split into individual digits and unknown UTF-8 characters fall back to byte-level decomposition, which improves robustness on technical and multilingual text. The 32K vocabulary was efficient for English but inflated token counts for non-Latin scripts; a Mandarin sentence might use 2-3x as many Llama 2 tokens as its English equivalent, and this was a major reason Llama 3 expanded to a 128K vocabulary.
The architecture does not use bias terms in the linear layers, a choice inherited from Llama 1 that slightly reduces parameter count and has been shown not to harm performance. The model uses tied input/output embeddings to save further parameters [1].
# Approximate hidden / feed-forward dimensions per variant
# (intermediate_size ~= 8/3 * hidden_size, rounded for hardware alignment)
LLAMA2_CONFIG = {
'7B': {'layers': 32, 'heads': 32, 'kv_heads': 32, 'hidden': 4096, 'ffn': 11008},
'13B': {'layers': 40, 'heads': 40, 'kv_heads': 40, 'hidden': 5120, 'ffn': 13824},
'34B': {'layers': 48, 'heads': 56, 'kv_heads': 8, 'hidden': 7168, 'ffn': 22016},
'70B': {'layers': 80, 'heads': 64, 'kv_heads': 8, 'hidden': 8192, 'ffn': 28672},
}
Llama 2 was pretrained on 2 trillion tokens drawn from publicly available sources, a 40% increase over the 1.4 trillion tokens used for Llama 1 [1]. Meta did not disclose the exact composition of the training data but stated that it included "a new mix of publicly available online data" and that data was filtered to remove sites known to contain high volumes of personal information. The corpus explicitly excludes data from any of Meta's own products and services such as Facebook, Instagram, and WhatsApp.
The paper's Table 10 reports the language distribution of the pretraining corpus: roughly 89.7% English, with the remaining tokens spread across 27 other languages including German, French, Swedish, Chinese, Spanish, Russian, and Dutch, plus an "unknown" bucket containing programming languages and other text without an identifiable natural language [1]. The strong English bias is reflected in downstream performance, where non-English benchmarks lag substantially behind English ones.
The pretraining knowledge cutoff is September 2022, although some of the fine-tuning data extends to July 2023 [12]. The data was processed with the same SentencePiece BPE tokenizer used in Llama 1, with a 32,000-token vocabulary [1].
All models were trained with a standard autoregressive language-modeling objective using the AdamW optimizer with beta1 = 0.9, beta2 = 0.95, and weight decay of 0.1 [1]. The learning rate followed a cosine schedule with 2,000 warmup steps, decaying to 10% of the peak value. Peak learning rates were 3.0 x 10^-4 for the 7B and 13B variants and 1.5 x 10^-4 for the 70B. Gradient clipping was set to 1.0, and the global batch size was 4 million tokens for all variants.
Training was conducted on Meta's Research Super Cluster (RSC) and on internal production clusters, using NVIDIA A100-80GB GPUs [12]. The total compute and carbon impact were disclosed at unusual granularity in the model card:
| Model | GPU Hours (A100-80GB) | Power per GPU | CO2 Emissions |
|---|---|---|---|
| Llama 2 7B | 184,320 | 400 W | 31.22 tCO2eq |
| Llama 2 13B | 368,640 | 400 W | 62.44 tCO2eq |
| Llama 2 70B | 1,720,320 | 400 W | 291.42 tCO2eq |
| All variants combined | 3,311,616 | 400 W | 539.00 tCO2eq |
Meta reported that 100% of these emissions were offset through its sustainability program. At a wholesale rate of roughly $1 per A100 GPU-hour in mid-2023, the 70B run alone implied a compute cost north of $1.7 million in commodity terms, although the actual cost to Meta on owned hardware would have been substantially lower.
| Aspect | Llama 1 (Feb 2023) | Llama 2 (Jul 2023) |
|---|---|---|
| Training tokens | 1.4 trillion | 2.0 trillion |
| Context length | 2,048 | 4,096 |
| Largest released model | 65B | 70B |
| Attention (largest) | MHA | GQA (8 KV heads) |
| Tokenizer vocab | 32,000 (BPE) | 32,000 (BPE, identical) |
| Commercial license | Research only | Yes, with 700M MAU clause |
| RLHF alignment | None | Yes (SFT + 5 RLHF rounds) |
| Disclosed compute | A100 hours not detailed | 3.31M A100-hours total |
The 40% increase in training data was one of the most impactful changes. Scaling laws research has consistently shown that training on more tokens improves model quality at a fixed parameter count, and the jump from 1.4T to 2T tokens produced measurable gains across benchmarks even at the same architecture and parameter count.
The Llama 2-Chat models underwent an extensive alignment process that combined supervised fine-tuning (SFT) with reinforcement learning from human feedback. The pipeline was described in unusual detail in the paper, making it one of the most transparent published accounts of an industrial-scale RLHF run [1]. Researchers and engineers at other labs treated the 76-page paper as a de facto recipe for replicating large-model alignment.
The first stage involved supervised fine-tuning on 27,540 high-quality prompt-response pairs written by human annotators [1]. Meta found that a relatively small number of carefully curated examples was more effective than larger sets of lower-quality data, and explicitly cautioned that "quality is all you need" for the SFT stage. The team reported that they began curating their own SFT data after observing that some publicly available instruction-tuning datasets contained noisy or generic responses.
SFT was run for two epochs with a cosine learning rate schedule peaking at 2 x 10^-5, weight decay 0.1, batch size 64, and a 4,096-token sequence length. Prompts and answers were concatenated with a special token in between, and loss was computed only over the answer tokens.
Meta trained two separate reward models on top of the SFT checkpoint, an unusual design choice at the time:
Using two reward models rather than a single combined one allowed Meta to manage the well-known tension between safety and helpfulness, where overly cautious models refuse legitimate requests and overly helpful models generate unsafe content. The two scores were combined at PPO time using a piecewise function that weighted the safety score more heavily when responses fell below a safety threshold [1].
The reward models were trained on human preference data collected through a process in which annotators compared pairs of model responses and selected the one they preferred. The annotation effort was substantial: by the time of the paper, Meta had collected 1,418,091 internal binary comparisons, supplemented by seven publicly available preference datasets bringing the total above 2.9 million comparisons [1]. Average dialog depths varied between 1.0 and 3.9 turns depending on source.
The RLHF process was iterative, spanning five successive versions (RLHF-V1 through RLHF-V5). Each iteration refined the model's behavior based on updated reward models and new preference data collected from the latest checkpoint. Meta employed two complementary techniques:
Meta applied rejection sampling fine-tuning for the first four rounds, then followed with PPO in the fifth round. This sequential combination allowed the model to benefit from both the broad quality improvements of rejection sampling and the targeted optimization of PPO [1]. Notably, only the 70B model used rejection sampling at full strength; smaller models inherited responses from the 70B teacher in a form of sequence-level distillation, which the paper credits with closing much of the capability gap between the 7B/13B chat models and the 70B chat model.
Meta introduced a technique called Ghost Attention (GAtt) to help the model follow system-level instructions consistently throughout a multi-turn conversation [1]. Without GAtt, chat models tend to drift away from the system prompt as the conversation grows longer, since the system message becomes a smaller fraction of the visible context. GAtt works by synthetically inserting the system message at multiple turns during training while masking out its tokens in the loss, teaching the model to attend to the original instruction even when many turns separate it from the current generation. The paper reports that GAtt produced near-perfect adherence to constraints (such as "always reply in haiku" or "never mention apples") for at least 20 turns, compared to roughly 4 turns without GAtt.
The default system prompt used for Llama 2-Chat establishes the model's intended behavior:
"You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature." [20]
Llama 2-Chat follows a specific INST/SYS token format defined by Meta:
<s>[INST] <<SYS>>
You are a helpful, respectful and honest assistant.
<</SYS>>
What is the capital of France? [/INST] The capital of France is Paris. </s>
The verbose default prompt drew criticism in the months after release for producing models that refused benign requests (for example, refusing to give a recipe for killing a process in Linux because it pattern-matched on the word "kill"). Meta acknowledged this behavior in the paper as the safety/helpfulness trade-off, and community fine-tunes such as those by NousResearch and Eric Hartford became popular partly because they used less restrictive system prompts. Mixing up the chat format silently degraded quality, and was a common source of bug reports in the weeks after release. Hugging Face's tokenizer.apply_chat_template and llama.cpp's --chat-template flag eventually standardized formatting.
Llama 2 demonstrated strong performance across academic benchmarks, consistently outperforming other open-source models available at the time of release. The paper reports both standard pretrained-model benchmarks and human evaluations of the chat variants.
Numbers below are taken directly from the Llama 2 paper unless otherwise noted [1].
| Benchmark | Llama 2 7B | Llama 2 13B | Llama 2 70B | Llama 1 65B | GPT-3.5 | GPT-4 |
|---|---|---|---|---|---|---|
| MMLU (5-shot) | 45.3 | 54.8 | 68.9 | 63.4 | 70.0 | 86.4 |
| GSM8K (8-shot) | 14.6 | 28.7 | 56.8 | 50.9 | 57.1 | 92.0 |
| HumanEval (pass@1) | 12.8 | 18.3 | 29.9 | 23.7 | 48.1 | 67.0 |
| TruthfulQA (% true & info) | 33.3 | 41.9 | 50.2 | 43.4 | 47.0 | n/a |
| BIG-Bench Hard (3-shot) | 32.6 | 39.4 | 51.2 | 44.5 | n/a | n/a |
| AGIEval | 21.8 | 28.5 | 40.0 | 31.6 | n/a | n/a |
| TriviaQA (1-shot) | 72.1 | 79.6 | 85.0 | 84.6 | n/a | n/a |
| NaturalQuestions (1-shot) | 25.7 | 31.8 | 33.0 | 32.5 | n/a | n/a |
The 70B model achieved 68.9 on MMLU, approaching GPT-3.5's 70.0 and improving over Llama 1's 65B by about 5.5 points [1]. On mathematical reasoning, the gap was even larger: GSM8K jumped from 50.9 (Llama 1 65B) to 56.8 (Llama 2 70B). Code generation (HumanEval) remained a relative weakness at 29.9, well below GPT-3.5's 48.1, and the gap to GPT-4 (67.0) was even larger. Code Llama was released a month later partly to close this gap [15].
For the chat variants, Meta conducted human evaluations on roughly 4,000 prompts spanning helpfulness and safety, with three annotators per prompt [1]. The headline result was that Llama 2-Chat 70B was statistically tied with ChatGPT (gpt-3.5-turbo, March 2023 snapshot) on helpfulness, with a 36% win rate, 31.5% tie rate, and 32.5% loss rate against ChatGPT.
Independent third-party evaluations followed quickly. LMSYS's Chatbot Arena, an Elo-style human preference ranking, placed Llama 2-Chat 70B in the top tier among open-weight models for the rest of 2023 [21]. MT-Bench, a multi-turn evaluation harness developed at LMSYS, reported a score of 6.86 for Llama 2-Chat 70B compared to GPT-4's 8.99 and ChatGPT's 7.94 [21]. AlpacaEval, an automated win-rate benchmark, scored the same model at 92.7% relative to text-davinci-003 [22]. These third-party numbers were broadly consistent with Meta's internal reports and helped legitimize the comparison with closed models.
The alignment process produced large improvements on safety benchmarks [1]:
The paper devoted nearly as much space to safety methodology as to capability training, including extensive red-teaming with both internal and external annotators. More than 350 people participated in adversarial red-teaming exercises, generating around 2,000 adversarial prompts that were used to evaluate and refine the safety reward model [1].
The Llama 2 Community License was one of the most consequential aspects of the release. Unlike Llama 1 (which was research-only), Llama 2 was released under a bespoke license that explicitly permitted commercial use [23]. The key terms included:
The license was not technically open source by the Open Source Initiative's definition, because it imposed both use restrictions (the Acceptable Use Policy) and the 700M MAU threshold. The OSI publicly stated in 2023 that the Llama 2 license did not meet the Open Source Definition, writing that "the Llama 2 license is not Open Source" because it places restrictions on commercial use and on the fields of endeavor in which the software can be used [26]. Critics, including the policy think tank Open Future, argued that calling Llama 2 "open source" was misleading because Meta did not release training data and provided only partial transparency about the data pipeline [27].
Meta and others in the industry countered that the license was far more permissive than anything previously offered at this model quality, and that "open weights" was the more accurate descriptor. The terminological debate sharpened in 2024 when the OSI published a formal Open Source AI Definition (OSAID) that required, among other things, sufficient information to recreate the model from scratch, a bar that no commercial frontier model met as of 2026.
Regardless of the debate, the practical effect was transformative. Thousands of developers and companies began building on Llama 2 within weeks of release, and the license became a template that influenced subsequent open-weight releases, notably the very similar Llama 3, 3.1, 3.2, 3.3, and Llama 4 community licenses that followed.
Meta designated Microsoft as the "preferred partner" for Llama 2, and the two companies announced their expanded AI partnership on the same day as the model release [2][3]. The partnership had several components:
The Microsoft partnership was unusual because Microsoft was simultaneously the primary backer and largest investor in OpenAI, which operated a closed-model strategy. By partnering with Meta on open models, Microsoft hedged its position, ensuring that Azure customers could access both closed (OpenAI) and open (Meta) ecosystems through the same control plane [28].
Llama 2's release had an outsized impact on the open AI ecosystem and is widely credited as the catalyst for the modern open-weight LLM movement [4][5]. Press coverage at the time emphasized the strategic significance of a major incumbent releasing high-quality models commercially, with TechCrunch describing Llama 2 as "Meta's broadside against OpenAI" and The Verge framing it as "the next phase of Meta's open-source AI push" [29][30].
Llama 2 became the most fine-tuned foundation model of 2023. By the time Llama 3 was announced in April 2024, more than 60,000 derivative models based on the Llama family had been uploaded to Hugging Face [4]. Below is a non-exhaustive sample of notable Llama 2-based derivatives. (Note: Vicuna v1 and Alpaca were Llama 1-based; only Vicuna v1.5 onward used Llama 2 as a base.)
| Derivative | Creator | Base | Notes |
|---|---|---|---|
| Vicuna v1.5 | LMSYS | Llama 2 7B/13B | ShareGPT-style dialogues, 4K and 16K context variants |
| WizardLM-2 (early) | WizardLM team | Llama 2 7B/13B/70B | Evol-Instruct synthetic data, strong MT-Bench scores |
| Nous-Hermes-2 | NousResearch | Llama 2 13B/70B | GPT-4 distilled instructions |
| OpenChat 3.5 | OpenChat team | Llama 2 7B | C-RLFT fine-tuning, MT-Bench above 7.0 |
| Tulu 2 | AI2 | Llama 2 7B/13B/70B | Diverse instruction mix with DPO alignment |
| Llama-2-7B-32K | Together AI | Llama 2 7B | RoPE rescaling for 32K context |
| Code Llama variants | Meta | Llama 2 7B/13B/34B/70B | Official code specializations |
| MedAlpaca, Meditron | Stanford / EPFL | Llama 2 7B/13B/70B | Medical question answering |
| LLaVA-1.5 | UW-Madison / Microsoft | Llama 2 7B/13B | Visual instruction tuning, image inputs |
| Llama Guard | Meta | Llama 2 7B | Input/output safety classifier shipped with Purple Llama |
A common misconception is that Mistral 7B (released September 2023) was a Llama 2 fine-tune. It was not. Mistral 7B was trained from scratch but used a Llama-style architecture (decoder-only transformer with RMSNorm, SwiGLU, RoPE) and was distributed under the Apache 2.0 license [31]. The architectural lineage from Llama 1 to Llama 2 to Mistral 7B illustrates how Meta's design choices became the de facto reference for the entire open-weight LLM ecosystem, even where the actual weights were not derived from Llama.
Many of these derivatives took the top spots on the Hugging Face Open LLM Leaderboard during late 2023 and early 2024, and they collectively defined what "open-weight chatbot" meant in this period. The community also developed runtime tools that made Llama 2 unusually portable, including llama.cpp (Georgi Gerganov's C++ port that runs the 7B on CPUs and Apple silicon), GGML/GGUF quantization formats, ExLlama, vLLM, and Text Generation Inference [13].
In December 2023, Meta launched Purple Llama, an umbrella initiative for open trust-and-safety tools built on top of Llama 2 [32]. The initial release included Llama Guard, a 7B classifier fine-tuned to detect unsafe content in both inputs and outputs against a six-category taxonomy (violence, sexual content, criminal planning, weapons, regulated substances, and self-harm). Purple Llama also shipped CyberSecEval, a benchmark suite for testing whether code-generating LLMs produce insecure code or assist in cyber-offense tasks. The Purple Llama project was significant because it gave deployers a way to compose a model with an open safety filter rather than relying solely on a proprietary moderation API.
In September 2023, Meta researchers published a follow-up paper, "Effective Long-Context Scaling of Foundation Models" (arXiv:2309.16039), describing Llama 2 Long, a continuation pretraining run that extended the context window from 4K to 32K tokens [33]. Llama 2 Long modified the RoPE base frequency and trained on an additional 400 billion tokens with longer sequences. Although the weights were not publicly released, the paper became influential as a recipe for context extension and informed both Code Llama's long context and later Llama 3 work.
The commercial license established a precedent that other model developers followed. Mistral AI's decision to release Mistral 7B and Mixtral under the Apache 2.0 license, and the broader trend toward open-weight releases from companies like Alibaba (Qwen), 01.AI (Yi), and DeepSeek, were all influenced by Llama 2's demonstration that open distribution could be commercially viable. By 2025 a majority of widely-used non-frontier LLMs were distributed under either Apache 2.0 or a Llama-style community license.
Meta reported that Llama usage (across all versions) grew 10x from January to July 2024, with token volume among major cloud providers more than doubling between May and July 2024 [4]. Cumulative downloads of Llama models passed 400 million by July 2024 and exceeded 600 million by early 2025. Meta also issued more than $2 million in Llama Impact Grants and Awards to support community projects.
Serving the 70B chat model in production typically required at least two NVIDIA A100-80GB GPUs in fp16, or one A100 in 4-bit quantization. Together AI, Anyscale, and Replicate published price points of roughly $0.65 to $1.00 per million output tokens for Llama 2-Chat 70B in the months after release, which was about 30% of GPT-3.5-turbo's then-current rate and made Llama 2 attractive for high-volume use cases. The 7B chat variant ran on a single consumer GPU (an RTX 3090 or 4090) at acceptable latency, and the GGUF-quantized version of 7B chat ran on Apple silicon laptops at 8 to 30 tokens per second, depending on quantization level.
Llama 2's release prompted immediate policy discussion. In the United States, the model became a reference case in debates over how the Biden administration's October 2023 Executive Order on AI should treat dual-use foundation models with widely available weights. The National Telecommunications and Information Administration (NTIA) ran a public comment period in early 2024 specifically asking whether "open foundation model weights" should be subject to additional reporting or restriction. In the United Kingdom, the AI Safety Institute used Llama 2-Chat as one of its early reference models for evaluation methodology. Within the European Union, Llama 2 was cited during AI Act trilogue negotiations as an example of a model whose distribution would not fit neatly into either the "general-purpose AI model" or the "high-risk AI system" categories then being drafted.
On August 24, 2023, roughly five weeks after the Llama 2 release, Meta introduced Code Llama, a family of code-specialized language models built on the Llama 2 foundation [15]. Code Llama was created by further training the base Llama 2 models on 500 billion tokens of code and code-related data, with additional long-context fine-tuning.
| Model | Parameters | Specialization | FIM | Context | Release |
|---|---|---|---|---|---|
| Code Llama 7B | 7B | General code | Yes | 16,384 | Aug 2023 |
| Code Llama 13B | 13B | General code | Yes | 16,384 | Aug 2023 |
| Code Llama 34B | 34B | General code | No | 16,384 | Aug 2023 |
| Code Llama 70B | 70B | General code | No | 16,384 | Jan 2024 |
| Code Llama - Python | 7-70B | Python-specific | Mixed | 16,384 | Aug 2023 / Jan 2024 |
| Code Llama - Instruct | 7-70B | Instruction-following | Mixed | 16,384 | Aug 2023 / Jan 2024 |
Three variant types were released for each size:
The 7B and 13B variants supported fill-in-the-middle (FIM) capability, allowing them to insert code into existing files given surrounding context, which made them suitable for IDE autocomplete. All Code Llama models supported a 16,384-token context window, four times that of base Llama 2, achieved through additional long-context fine-tuning with adjusted RoPE frequencies (theta = 1,000,000 instead of 10,000) [15].
Code Llama 34B scored 53.7 on HumanEval (pass@1), almost double the base Llama 2 70B's 29.9, demonstrating the value of domain-specific continued pretraining. The 70B variant added in January 2024 reached 67.8 pass@1, briefly making it the highest-scoring open-weight code model. Code Llama also seeded a wave of community fine-tunes including Phind-CodeLlama-34B, WizardCoder-34B from Microsoft Research, and DeepSeek's first code model (which began life as a Code Llama derivative before its team retrained from scratch). The released-but-not-flagship 34B parameter point in the Code Llama lineup partially compensated for Meta's decision not to release a 34B base text model.
Llama 2 was succeeded by Llama 3, announced on April 18, 2024 [34]. Llama 3 was initially released in 8B and 70B sizes (replacing the 7B/13B/70B lineup) and made several major changes:
The following table summarizes the key generational differences:
| Aspect | Llama 2 (Jul 2023) | Llama 3 (Apr 2024) |
|---|---|---|
| Released sizes | 7B, 13B, 70B | 8B, 70B (later 405B in 3.1) |
| Training tokens | 2 trillion | 15+ trillion |
| Tokenizer vocab | 32,000 (SentencePiece BPE) | 128,256 (tiktoken-based) |
| Native context | 4,096 | 8,192 (128K in 3.1) |
| Attention | MHA on 7B/13B, GQA on 70B | GQA at all sizes |
| 70B MMLU | 68.9 | approximately 82 |
The Llama 3 family was subsequently expanded by:
Llama 3 70B at release scored approximately 82 on MMLU compared to Llama 2 70B's 68.9, a roughly 13-point jump that effectively closed the gap to GPT-4-class closed models [34].
As of early 2026, Llama 2 weights remain freely downloadable from Meta and from Hugging Face under the original Llama 2 Community License, and are still used in some production systems, particularly where regulatory or supply-chain audits favor a well-understood older model over a frontier one. The 7B variant in particular continues to appear in research papers as a standard baseline, and llama.cpp's GGUF distribution of Llama 2 7B chat is one of the most-downloaded GGUF files on Hugging Face. For new deployments, however, the Llama 3 and Llama 4 families offer substantially better performance across all benchmarks, and most active users have migrated.
Llama 2's most lasting contribution is not the models themselves but the precedent they set. By demonstrating that a major technology company could release high-quality models under a permissive license and still benefit strategically, Meta shifted the industry's expectations about openness. The model proved that open and commercial were not opposing goals, and that proof has continued to shape how AI models are developed, distributed, and regulated. Almost every subsequent major open-weight release, including Mistral, Mixtral, Qwen, Yi, DeepSeek, Phi, Gemma, and Meta's own Llama 3 and 4, followed templates that Llama 2 either established directly (community-style commercial licenses) or made viable (publishing detailed alignment recipes).