Gemma

Gemma is a family of open-weight models developed by Google DeepMind. The family began in February 2024 with text-only decoder models derived from research used for Google's Gemini systems, then expanded to vision, audio, long-context, edge-oriented, mixture-of-experts, embedding, translation, medical, safety, and interpretability releases.^[1][2] "Gemma" is therefore a family name rather than one architecture or one license.

Through July 28, 2026, the core generations were Gemma 1, Gemma 2, Gemma 3, Gemma 3n, and Gemma 4. Gemma 1 through Gemma 3n and listed derivatives are distributed under Google's custom Gemma Terms and Prohibited Use Policy. Gemma 4 is distributed under the Apache License 2.0.^{[14][15][16][17]} The family-wide description "open-weight" is more precise than treating every Gemma release as equivalent under a single definition of open source.

Scope and release history

Google's release log separates the date weights became available from the date of a launch announcement. It also shows that some generations grew after their initial release. In particular, the Gemma 2 2B model followed the 9B and 27B models by about a month, and Gemma 4 12B Unified followed the initial Gemma 4 set.^[1]

Scroll sideways for more →

The letters and rounded numbers in model names are not always total parameter counts. Gemma 3n E2B and E4B contain about 5B and 8B parameters in total, while their core transformer computation is approximately 2B and 4B. Gemma 4 uses the same "effective" convention for E2B and E4B. Its 26B-A4B model has about 25.2B total language-model parameters and 3.8B active parameters per token. The technical report's detailed table separately accounts for modality encoders and speculative-decoding drafters, so a marketed size should not be used as an exact sum of every distributed component.^[8][13]

Gemma 1

Gemma 1 introduced pretrained and instruction-tuned 2B and 7B text models. Both are decoder-only transformers with an 8,192-token context, a 256,128-entry SentencePiece vocabulary, rotary position embeddings, GeGLU feed-forward blocks, and RMSNorm. The 2B model has 18 layers and uses multi-query attention with one key-value head; the 7B model has 28 layers and uses 16-head multi-head attention.^[3]

Scroll sideways for more →

The technical report describes the pretraining mixture only at a category level: primarily English web documents, mathematics, and code. It says the models were not multimodal and were not trained specifically for leading multilingual performance. Google reported filtering for unsafe or sensitive material, evaluation-set decontamination, and measures intended to reduce recitation, but it did not publish a complete training corpus.^[3]

The instruction-tuned checkpoints were produced with supervised fine-tuning and reinforcement learning from human feedback. Their safety evaluation covered several categories, but the report explicitly warns that testing cannot cover every application or deployment scenario.^[3]

Gemma 2

Gemma 2 kept a decoder-only text architecture and an 8,192-token context while changing the attention and normalization design. All three sizes use grouped-query attention. Layers alternate between a 4,096-token local sliding window and global attention spanning 8,192 tokens. The design also adds pre-norm and post-norm RMSNorm and caps attention and output logits with a scaled hyperbolic tangent.^[5]

Scroll sideways for more →

The 2B and 9B models were trained with knowledge distillation from larger teachers, whereas the 27B model was trained from scratch. The report describes distillation as matching the teacher's token-probability distribution rather than learning only from one next-token label. It also says the smaller models were trained on more than 50 times the token count predicted by the compute-optimal calculation used in the paper. This is a statement about that scaling analysis, not a universal definition of overtraining.^[5]

Gemma 2's data remained primarily English and included web documents, code, and science articles. The instruction-tuning pipeline used supervised examples, preference data, reinforcement learning from human feedback, and model merging. Google's launch post on June 27 covered the 9B and 27B weights; its release log dates the 2B release to July 31.^[1][4][5]

Gemma 3

Gemma 3 added image input and longer context. The 1B model is text-only with a 32K context. The 4B, 12B, and 27B models combine a language model with a roughly 400M-parameter SigLIP vision encoder and support a 128K context. The encoder processes images at 896 by 896 pixels. An optional inference-time Pan and Scan procedure divides images with difficult aspect ratios or fine details into additional crops.^[6][7]

The language architecture uses grouped-query attention, RMSNorm, and query-key normalization. It interleaves five local-attention layers for every global-attention layer. Local attention spans 1,024 tokens, while global layers cover the long context. Global layers use a higher RoPE base frequency than local layers. This arrangement reduces the growth of key-value cache memory relative to using global attention in every layer.^[7]

Scroll sideways for more →

The Gemma 3 report says the pretraining mix included text and images, increased multilingual data, and used knowledge distillation. Google described support for more than 140 pretraining languages and more than 35 languages with out-of-the-box support, but those phrases refer to different thresholds and should not be treated as equivalent guarantees of quality.^[6][7] The published vocabulary has about 262,000 entries, replacing the 256,128-entry tokenizer used in Gemma 1 and Gemma 2.^[7]

Quantization-aware-trained checkpoints accompanied the raw checkpoints. The report evaluates several weight representations and reports memory figures under specified formats and context lengths. Actual memory use depends on the checkpoint format, runtime, context, cache precision, accelerator, and surrounding software, so those tables are measurements under a protocol rather than universal device requirements.^[7]

Gemma 3n

Gemma 3n is an edge-oriented branch with E2B and E4B models. Its MatFormer design trains a smaller E2B submodel within the E4B model, and its Per-Layer Embeddings move part of the parameter storage and computation outside the accelerator-resident transformer core. Google reports total counts of about 5B and 8B but effective accelerator sizes of about 2B and 4B.^[8]

The models accept text, images, video frames, and single-channel audio and generate text. The model card specifies a 32K total input context, images encoded at 256, 512, or 768 pixels, and audio encoded at 6.25 tokens per second. It reports approximately 11 trillion training tokens, a June 2024 knowledge cutoff, and content in more than 140 languages.^[9]

Gemma 3n uses a MobileNet-V5-based vision encoder and a Universal Speech Model-based audio encoder. Google's release measurements include operation with as little as about 2 GB for E2B and 3 GB for E4B, and about twice the prefill performance of Gemma 3 4B from key-value cache sharing. Those figures are first-party results for specific implementations; they should not be read as guarantees on arbitrary phones or runtimes.^[8]

Gemma 3 270M and FunctionGemma

Gemma 3 270M is a text model released for narrow, task-specific fine-tuning. Google divides its 270M parameters into about 170M embedding parameters and 100M transformer-block parameters and released pretrained, instruction-tuned, and quantization-aware-trained checkpoints. Google's Pixel 9 Pro test reported that an INT4 checkpoint used 0.75 percent of battery capacity across 25 conversations, but the announcement does not establish that result for other prompts, devices, or runtimes.^[10]

FunctionGemma, released in December 2025, is a 270M instruction-tuned derivative intended as a starting point for function calling. It formats natural-language requests as tool or API actions and is designed to be further fine-tuned for a defined action space. Google's own example improved after domain-specific fine-tuning, which is evidence for that test set rather than a claim that the base checkpoint reliably controls arbitrary software.^[11]

Gemma 4

Google's release log dates the initial Gemma 4 weights to March 31, 2026; the launch post and Apache 2.0 announcement are dated April 2. The initial lineup included dense E2B, E4B, and 31B models plus the sparse 26B-A4B mixture-of-experts model. Gemma 4 12B Unified was added on June 3.^[1][12][14]

Gemma 4 is natively multimodal and adds a "thinking" mode in which an instruction-tuned checkpoint can emit a reasoning trace before its answer. E2B and E4B use separate image and audio encoders. The 26B-A4B and 31B models use image encoders but do not include the audio encoder described for the edge models. The 12B model instead projects raw image patches and 40-millisecond audio chunks into the language-model embedding space without a separate pretrained vision or audio encoder.^[13]

Scroll sideways for more →

The small models use four local-attention blocks per global block, while the other sizes use five local blocks per global block. Gemma 4 also uses partial RoPE in global attention, key-value cache reuse or sharing in several sizes, quantization-aware training, and separately released multi-token prediction drafters for speculative decoding. The technical report describes pretraining data from web documents, code, images, and audio for the audio-capable sizes, with a January 2025 data cutoff.^[13]

Google reported more than 400 million cumulative downloads and more than 100,000 community variants at the April 2026 launch. These are first-party cumulative figures and do not identify a counting methodology, unique users, or how repeated downloads were handled.^[12]

Training, post-training, and evaluation

Across generations, Google discloses broad source categories, token totals for many checkpoints, filtering goals, and training infrastructure, but not a complete itemized training dataset. The reports describe attempts to remove some personal or unsafe material, decontaminate benchmarks, and reduce recitation. These measures lower specific measured risks; they do not prove that every training item was lawfully sourced, that evaluation contamination is absent, or that outputs cannot reproduce sensitive material.^{[3][5][7][13]}

Pretrained checkpoints predict continuations, while instruction-tuned checkpoints receive additional supervised and preference-based training. The exact post-training recipe changed by generation. Gemma 1 and Gemma 2 used supervised fine-tuning and reinforcement learning from human feedback; Gemma 3 added distillation from a larger instruction-tuned teacher and reinforcement-learning stages; Gemma 4 added a thinking mode and function-calling formatting.^{[3][5][7][13]}

Benchmark scores in the technical reports are vendor-run results under stated prompts, shot counts, normalization choices, model revisions, and inference settings. Chatbot Arena uses blind pairwise user preferences and converts comparisons into ratings; its scores change as votes and model pools change.^[31] For that reason, a claim that a Gemma model was at a particular rank is only meaningful with a date and protocol.

Independent results also show why parameter labels alone are insufficient for deployment choices. A 2026 preprint compared seven Gemma 4, Phi-4, and Qwen3 checkpoints on four tasks and three prompting strategies while measuring accuracy, latency, and peak GPU memory. In that setup, Gemma 4 E4B had the best weighted result under one prompting condition, while Gemma 4 26B-A4B used much more memory despite activating fewer parameters. The authors concluded that sparse activation alone did not determine the best operating point. The finding is bounded to their hardware, tasks, checkpoints, and prompts.^[32]

Specialized models and research tools

The Gemma name also covers task-specific models and research artifacts. These are not all interchangeable chat models, and several are derived from different core generations.

Scroll sideways for more →

CodeGemma's report documents a 2B code-completion model and 7B pretrained and instruction-tuned checkpoints.^[19] RecurrentGemma replaces most global transformer attention with Griffin's gated linear recurrence and local attention, giving inference memory that does not grow with the full sequence in the same way as a standard global key-value cache.^[20] PaliGemma combines a SigLIP image encoder with a Gemma 2B language model and was designed as a base for transfer to vision-language tasks rather than as a general chat assistant.^[21]

ShieldGemma's first release used Gemma 2 checkpoints as text safety classifiers.^[22] The separate ShieldGemma 2 release is an image safety classifier based on Gemma 3 4B.^[6] Gemma Scope is also generation-specific: the original release targeted Gemma 2, whereas Gemma Scope 2 targeted Gemma 3. Calling Scope 2 an expansion to Gemma 2 reverses that chronology.^[23][24]

Later research branches adapted Gemma to other model forms and domains. T5Gemma explored encoder-decoder adaptation from Gemma 2.^[25] EmbeddingGemma used a Gemma 3 foundation to produce a compact text embedding model.^[26] TranslateGemma adapted Gemma 3 sizes for translation.^[27] MedGemma added medical text and image specialization to Gemma 3 4B and 27B.^[28] VaultGemma trained a 1B checkpoint with differential privacy using the Gemma 2 data mixture.^[29] Each has its own evaluation protocol and intended use; none should inherit a general claim of clinical, privacy, or translation suitability without task-specific validation.

Distribution and deployment

Google publishes official Gemma implementations, examples, and links to weights in its GitHub repository and documentation. Weights have been distributed through Kaggle and Hugging Face, while Google documents integrations with Keras, JAX, PyTorch-based Transformers, Gemma.cpp, LiteRT-LM, and multiple third-party runtimes.^[1][30] Support varies by generation, quantization, modality, and model size.

Local execution does not imply a fixed hardware requirement. Stored weights, temporary activations, key-value cache, image or audio encoders, runtime overhead, batch size, prompt length, and quantization all affect memory and throughput. Vendor statements such as "runs on a single GPU" or "as little as 2 GB" describe particular configurations, not every checkpoint at its advertised size.^[7][8][13]

Running a model on-device can keep inputs local if the surrounding application performs no network transfer. That property comes from the deployment architecture, not from the model weights alone. Applications must still evaluate logging, telemetry, tool calls, retrieved data, and operating-system services before making privacy claims.

Licensing and open-model terminology

The Gemma Terms apply to listed Gemma 1, Gemma 2, Gemma 3, Gemma 3n, and related model derivatives. They grant rights to use, reproduce, modify, distribute, and create derivatives subject to conditions, including passing required restrictions to recipients. The separate Prohibited Use Policy restricts specified harmful uses. The terms page explicitly directs Gemma 4 users to the Gemma 4 license.^[15][16]

Gemma 4 is licensed under Apache License 2.0. That license grants copyright and patent permissions subject to notice, attribution, and other conditions in the license text.^[14][17] This is a material licensing change from earlier Gemma generations, so a downstream product must identify which checkpoint and license it uses. This article is descriptive and is not legal advice.

The Open Source Initiative's Open Source AI Definition evaluates more than whether weights can be downloaded. It calls for the freedoms to use, study, modify, and share, together with the preferred form for making modifications, including sufficient data information, training code, and parameters.^[18] Earlier Gemma releases carry use restrictions, and Google does not publish the complete training corpora for any core generation. "Open-weight" therefore remains the least ambiguous description for the family as a whole, even though Google and others also use "open model" and describe Gemma 4's Apache license as open source.

Safety and limitations

Gemma checkpoints can generate false, outdated, biased, unsafe, or fabricated content. Their knowledge is bounded by their training data and cutoff; multimodal input does not guarantee reliable perception; long advertised context does not ensure that every detail is retrieved or reasoned over correctly. Smaller or quantized checkpoints may trade accuracy for memory, latency, or energy use.^{[3][5][7][13]}

Google reports filtering, red-team work, automated tests, and human evaluations, but every core technical report cautions that testing is incomplete. A model that has low measured policy-violation rates under one test can behave differently after fine-tuning, quantization, system integration, prompt changes, or tool access. Developers need use-case-specific evaluation, access controls, monitoring, and human review where errors could cause harm.^{[3][5][7][13]}

The family also presents reproducibility limits. Broad data categories and aggregate token totals do not permit an independent researcher to reconstruct the pretraining corpus. Vendor benchmark tables can be checked against published protocols, but proprietary training data, teacher models, preference datasets, and parts of the post-training pipeline prevent full reproduction.

See also

• Gemini
• Large language model
• Knowledge distillation
• Mixture of experts
• Quantization
• Mechanistic interpretability
• Edge computing

References

1. ^Google AI for Developers. "Gemma releases." Last updated July 2, 2026. ai.google.dev/...releases
2. ^Google. "Gemma: Introducing new state-of-the-art open models." February 21, 2024. blog.google/...gemma-open-models
3. ^Gemma Team, Google DeepMind. "Gemma: Open Models Based on Gemini Research and Technology." arXiv:2403.08295v1, 2024. arxiv.org/...2403.08295v1
4. ^Google. "Gemma 2 is now available to researchers and developers." June 27, 2024. blog.google/...google-gemma-2
5. ^Gemma Team, Google DeepMind. "Gemma 2: Improving Open Language Models at a Practical Size." arXiv:2408.00118v2, 2024. arxiv.org/...2408.00118v2
6. ^Google. "Gemma 3: The most capable model you can run on a single GPU or TPU." March 12, 2025. blog.google/...gemma-3
7. ^Gemma Team, Google DeepMind. "Gemma 3 Technical Report." arXiv:2503.19786v1, 2025. arxiv.org/...2503.19786v1
8. ^Google Developers Blog. "Introducing Gemma 3n: The developer guide." June 26, 2025. developers.googleblog.com/...ma-3n-developer-guide
9. ^Google AI for Developers. "Gemma 3n model card." ai.google.dev/...model_card
10. ^Google Developers Blog. "Introducing Gemma 3 270M: The compact model for hyper-efficient AI." August 14, 2025. developers.googleblog.com/...roducing-gemma-3-270m
11. ^Google. "FunctionGemma: Bringing bespoke function calling to the edge." December 18, 2025. blog.google/...functiongemma
12. ^Google. "Gemma 4: Byte for byte, the most capable open models." April 2, 2026. blog.google/...gemma-4
13. ^Gemma Team, Google DeepMind. "Gemma 4 Technical Report." arXiv:2607.02770v1, 2026. arxiv.org/...2607.02770v1
14. ^Google Open Source Blog. "Gemma 4: Expanding the Gemmaverse with Apache 2.0." April 2, 2026. opensource.googleblog.com/...averse-with-apache-20
15. ^Google. "Gemma Terms of Use." Last modified April 1, 2026. ai.google.dev/...terms
16. ^Google. "Gemma Prohibited Use Policy." Last modified February 21, 2024. ai.google.dev/...prohibited_use_policy
17. ^Apache Software Foundation. "Apache License, Version 2.0." January 2004. apache.org/...LICENSE-2.0
18. ^Open Source Initiative. "The Open Source AI Definition, Version 1.0." 2024. opensource.org/...open-source-ai-definition
19. ^CodeGemma Team. "CodeGemma: Open Code Models Based on Gemma." arXiv:2406.11409v1, 2024. arxiv.org/...2406.11409v1
20. ^Botev et al. "RecurrentGemma: Moving Past Transformers for Efficient Open Language Models." arXiv:2404.07839v1, 2024. arxiv.org/...2404.07839v1
21. ^Beyer et al. "PaliGemma: A Versatile 3B VLM for Transfer." arXiv:2407.07726v1, 2024. arxiv.org/...2407.07726v1
22. ^Zeng et al. "ShieldGemma: Generative AI Content Moderation Based on Gemma." arXiv:2407.21772v1, 2024. arxiv.org/...2407.21772v1
23. ^Lieberum et al. "Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2." arXiv:2408.05147v1, 2024. arxiv.org/...2408.05147v1
24. ^Google DeepMind. "Gemma Scope 2: Helping the AI safety community deepen understanding of complex language model behavior." December 19, 2025. deepmind.google/...complex-language-model-behavior
25. ^Zhang et al. "Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation." arXiv:2504.06225v1, 2025. arxiv.org/...2504.06225v1
26. ^Vera et al. "EmbeddingGemma: Powerful and Lightweight Text Representations." arXiv:2509.20354v1, 2025. arxiv.org/...2509.20354v1
27. ^Finkelstein et al. "TranslateGemma Technical Report." arXiv:2601.09012v1, 2026. arxiv.org/...2601.09012v1
28. ^Sellergren et al. "MedGemma Technical Report." arXiv:2507.05201v1, 2025. arxiv.org/...2507.05201v1
29. ^Sinha et al. "VaultGemma: A Differentially Private Gemma Model." arXiv:2510.15001v1, 2025. arxiv.org/...2510.15001v1
30. ^Google DeepMind. "Gemma: Open Models Based on Gemini Research and Technology." GitHub repository. github.com/...gemma
31. ^Chiang et al. "Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference." arXiv:2403.04132v1, 2024. arxiv.org/...2403.04132v1
32. ^Manik and Wang. "Gemma 4, Phi-4, and Qwen3: Accuracy-Efficiency Tradeoffs in Dense and MoE Reasoning Language Models." arXiv:2604.07035v1, 2026. arxiv.org/...2604.07035v1