Model card
A model card is a short, standardized document that accompanies a trained machine learning model and reports its intended use, training data, evaluation results across different population subgroups, ethical considerations, and known limitations. Model cards were introduced by Margaret Mitchell, Timnit Gebru, and colleagues at Google in the 2019 paper "Model Cards for Model Reporting," which defines them as "short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups" [1]. They function as a form of AI transparency documentation, analogous to nutrition labels for food, datasheets for electronic components, or Material Safety Data Sheets in chemistry, and have become a de facto industry standard largely through Hugging Face, where a model repository's README file serves as its model card across the platform's more than 2 million public models [1][3][24].
Model cards are part of a broader ecosystem of AI documentation practices that includes datasheets for datasets (Gebru et al., 2021), system cards (used by OpenAI for models like GPT-4), and regulatory documentation requirements under frameworks like the EU AI Act. Together, these documentation standards aim to improve accountability, reproducibility, and informed decision-making across the AI development lifecycle. They sit at the intersection of responsible AI, AI governance, and AI safety practice.
Who created model cards, and when?
The foundational paper, "Model Cards for Model Reporting," was published at the ACM Conference on Fairness, Accountability, and Transparency (FAT*, later renamed FAccT), held January 29-31, 2019 in Atlanta. The paper was authored by Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru, all of whom were affiliated with Google at the time. The first preprint was posted to arXiv (1810.03993) on October 5, 2018, with the revised version finalized January 14, 2019 [1].
Mitchell co-founded and co-led Google's Ethical AI team alongside Gebru. The model cards proposal grew out of that team's broader work on fairness, accountability, and bias in deployed machine learning. After Gebru's controversial departure from Google in December 2020, Mitchell was also fired in February 2021 following an internal investigation. She joined Hugging Face later that year as Chief Ethics Scientist, where she continued the model cards work and helped shape the platform's documentation practices [2][7].
The paper identified a gap in how machine learning models were documented and shared. While software engineering had established practices for documentation (API references, changelogs, README files), machine learning models were frequently released with minimal information about their intended use, performance across different populations, training data composition, or known limitations. This lack of transparency made it difficult for downstream users to evaluate whether a model was appropriate for their specific application, and it obscured potential harms to affected communities [1].
Mitchell et al. proposed model cards as a short, structured document accompanying any trained machine learning model. The paper drew on practices from other fields: datasheets that characterize electronic components under different test conditions, crash-test dummies with female characteristics, warning labels on food and toys, and the U.S. Food and Drug Administration's 1998 requirement that clinical trial results be disaggregated by groups such as age, race, and gender. The authors argued that a standardized documentation format would facilitate better communication between model developers and model users, encourage the evaluation of models across different demographic groups, and create accountability for known limitations [1].
The paper's most cited definition reads: "Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains." The emphasis on intersectional evaluation, drawing on critical race theory and earlier fairness work, was a deliberate departure from aggregate accuracy reporting [1].
The paper demonstrated the concept with example model cards for two Google models: a smile detector trained on the CelebA dataset and a toxicity classifier built on the Perspective API. The smile-detection card revealed higher false discovery rates for older men. The toxicity card surfaced bias against terms related to sexual orientation. In both cases the disaggregated reporting exposed disparities that aggregate accuracy numbers had hidden, demonstrating exactly the failure mode the authors were trying to address [1].
What does a model card contain?
The original Mitchell et al. framework specifies nine sections that a model card should include. These sections have been refined and expanded by subsequent adopters, but the core structure remains consistent across most modern templates [1].
| Section | Description | Example content |
|---|---|---|
| Model details | Basic information about the model | Name, version, type (e.g., classification, generation), developer, release date, license, citation, contact |
| Intended use | What the model was designed for | Primary use cases, intended users, out-of-scope uses |
| Factors | Variables that affect model performance | Demographic factors (age, gender, ethnicity, Fitzpatrick skin type), environmental factors (lighting, noise), instrumentation factors (camera type, microphone quality) |
| Metrics | How performance is measured | Accuracy, F1 score, BLEU, perplexity, fairness metrics, decision thresholds, variation approaches, disaggregated by relevant factors |
| Evaluation data | Datasets used for testing | Dataset name, size, composition, motivation for choice, any known biases |
| Training data | Datasets used for training | Sources, size, collection methodology, preprocessing steps, any known limitations |
| Quantitative analyses | Detailed performance results | Unitary results per factor, intersectional results, confidence intervals, error analysis |
| Ethical considerations | Known risks and societal impact | Potential for harm, sensitive use cases, populations at risk, mitigation strategies considered during development |
| Caveats and recommendations | Known limitations and usage guidance | Failure modes, conditions under which the model should not be used, recommended downstream evaluation |
The emphasis on disaggregated evaluation, where performance metrics are broken down across demographic or contextual subgroups rather than reported only as aggregate numbers, is one of the most important contributions of the model card framework. A face recognition model might achieve 99% accuracy overall but only 85% accuracy on darker-skinned faces; an aggregate metric would hide this disparity, while a model card would make it explicit. This approach was directly informed by Joy Buolamwini and Timnit Gebru's earlier Gender Shades research, which found commercial face analysis systems performed dramatically worse on darker-skinned women than on lighter-skinned men [1].
The paper also proposed two specific reporting modes: unitary results, which show performance per single factor, and intersectional results, which show performance at the intersection of factors (for example age x skin type). The intersectional view is more demanding but typically reveals worse outcomes than either factor alone would suggest, which is precisely why Mitchell et al. argued it was needed.
How does Hugging Face implement model cards?
Hugging Face, the open-source AI platform and model hosting service, has become the primary venue where model cards are created and consumed in practice. The Hugging Face Hub crossed one million hosted models in 2024 and surpassed two million in 2025, and passed 3 million in August 2026, with a repository's README.md file serving as its model card [3][4][24][26]. The first million models took more than 1,000 days to accumulate after the Hub began tracking in March 2022, while the second million arrived just 335 days later, a roughly 65% reduction in the time required, which AI World linked to the widespread adoption of small fine-tuned models [25].
Hugging Face's adoption of model cards has been the single most significant factor in their widespread use. When a researcher or organization uploads a model to the Hub, the platform prompts them to fill out a model card using a standardized template. This template draws on the original Mitchell et al. framework but has been expanded to include additional metadata fields relevant to the Hugging Face ecosystem, such as pipeline tags (text-classification, image-generation, etc.), language codes, datasets used, and library compatibility [3].
In 2022, Hugging Face launched the Model Card Guidebook, authored primarily by Ezi Ozoani, Marissa Gerchick, and Margaret Mitchell. The guidebook bundled five resources: an updated model card template in the huggingface_hub library, an Annotated Model Card Template explaining how to fill out each section, a user study on model card usage at the Hub, a landscape analysis of the broader documentation ecosystem, and a Model Card Creator Tool that allows users to generate model cards through a graphical interface without writing markdown directly. The release also added prompt text inside the template to encourage thorough completion of the Bias, Risks and Limitations section [3].
The quality of model cards on Hugging Face varies significantly. Models released by major organizations (Google, Meta, Microsoft, Mistral AI) typically have detailed, well-structured model cards. Community-contributed models, which make up the majority of Hub content, range from thorough documentation to completely empty README files. A 2024 analysis published in Nature Machine Intelligence by Liang et al. systematically examined 32,111 AI model cards on the Hub (drawn from 74,970 models). The study found that the Training section was the most consistently filled-out, while the Environmental Impact, Limitations, and Evaluation sections had the lowest completion rates. The same paper showed that improving model card completeness correlated with higher download rates, suggesting documentation pays off in adoption as well as accountability [5].
What metadata goes in a Hugging Face model card?
Hugging Face model cards include a YAML metadata block at the top of the README.md file, delimited by triple-dash lines. This block enables structured search, filtering, and integration with the platform's API. Key metadata fields include:
| Field | Purpose |
|---|---|
| language | ISO 639-1 codes for languages the model supports or was trained on |
| license | License identifier (apache-2.0, mit, llama3, etc.) |
| tags | Free-form tags for categorization and discovery |
| datasets | Hub datasets used for training or evaluation |
| metrics | Performance metrics and their reported values |
| pipeline_tag | The task type (text-generation, image-classification, automatic-speech-recognition, etc.) |
| library_name | The compatible ML library (transformers, diffusers, sentence-transformers); required for transformers repos created after August 2024 |
| base_model | The model this one was fine-tuned, adapted, or quantized from |
| thumbnail | Image used in social media previews |
| model-index | Structured evaluation results with task, dataset, metric, and source fields |
The model-index field deserves special mention because it lets the Hub surface evaluation numbers in a standardized way. A repo can declare, for example, that the model achieved 87.3 on MMLU using the lm-eval-harness, and that result will appear on aggregated leaderboards rather than buried in prose.
How do datasheets for datasets relate to model cards?
Model cards were designed to document models, but a parallel documentation framework exists for the datasets used to train and evaluate those models. "Datasheets for Datasets," proposed by Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daume III, and Kate Crawford, was first published as a preprint in March 2018 and later appeared in Communications of the ACM in December 2021 [6].
The datasheets framework takes its name from the electronics industry, where every component ships with a datasheet specifying its operating characteristics. Gebru et al. argued that every dataset used in machine learning should be accompanied by a similar document covering motivation, composition, the data collection process, any preprocessing or labeling steps, recommended uses, distribution, maintenance plans, and legal or ethical considerations. The full questionnaire contains 57 questions across seven categories, designed to be answered before a dataset is shared widely [6].
The key questions a dataset datasheet answers include:
| Category | Example questions |
|---|---|
| Motivation | Why was the dataset created? Who created it? Who funded it? |
| Composition | What do the instances represent? How many are there? Is there missing data? Does it contain confidential information? |
| Collection process | How was the data collected? Who was involved? Over what timeframe? Were individuals notified? Did they consent? |
| Preprocessing | Was any data cleaned, filtered, or labeled? What tools or procedures were used? |
| Uses | What tasks has the dataset been used for? Are there tasks it should not be used for? |
| Distribution | How is the dataset distributed? Under what license? Are there access restrictions? |
| Maintenance | Who maintains the dataset? How can errors be reported? Will it be updated? |
Datasheets for datasets and model cards are complementary: a model card references the training and evaluation data, while the dataset's datasheet provides the detailed provenance information for that data. Together, they create a documentation chain from raw data to deployed model. Hugging Face later integrated dataset cards (a YAML-fronted README on every dataset repo) that mirror this structure for the Hub [6].
Model cards versus datasheets at a glance
| Dimension | Model card | Datasheet for datasets |
|---|---|---|
| What it documents | A trained machine learning model | A dataset used for training or evaluation |
| Originating paper | Mitchell et al., 2019 (FAT*) | Gebru et al., 2018 (preprint), 2021 (CACM) |
| Core sections | 9 (Model details to Caveats) | 7 categories, 57 questions |
| Disaggregation focus | Performance across demographic groups | Composition and collection across population subgroups |
| Typical author | Model developer | Dataset curator or maintainer |
| Common venue | Hugging Face README, internal tooling | Dataset README, paper appendix |
How does a system card differ from a model card?
System cards represent an evolution of the model card concept, designed for AI systems that are more complex than a single model. The term was introduced by Meta AI in February 2022 and popularized by OpenAI, which has published system cards for several of its major releases, including GPT-4 (March 2023), GPT-4V (the vision-enabled variant, September 2023), GPT-4o (August 2024), the o1 reasoning models (December 2024), and subsequent frontier releases [8].
A system card differs from a model card in several respects. While a model card documents a single trained model in isolation, a system card documents the entire system that is deployed to users, including the base model, fine-tuning procedures, safety mitigations (such as RLHF training, content filters, and rate limits), deployment configuration, and the results of red-teaming and safety evaluations. System cards also typically include a discussion of the system's capabilities and the risks those capabilities pose in deployment [8].
OpenAI's GPT-4 System Card, published March 23, 2023 alongside the GPT-4 Technical Report, set the template that later system cards have followed. OpenAI engaged more than 50 external experts, with backgrounds in fairness, alignment research, dis/misinformation, chemistry, biorisk, cybersecurity, nuclear risks, and other domains, who probed early versions of the model over several months starting in August 2022. The card documented specific capabilities that raised safety concerns, including the model's ability to provide step-by-step instructions for synthesizing dangerous chemicals (mitigated through refusal training), its potential to identify private individuals when augmented with outside data, and its potential to lower the cost of some steps of a cyberattack. The card also described the limitations of the safety measures in place, including known jailbreaking techniques [8].
The GPT-4o System Card (August 2024) extended this to multimodal systems, adding voice safety evaluations covering unauthorized voice generation, speaker identification, disallowed audio content, and ungrounded inferences. The o1 System Card (December 2024) introduced new evaluations specific to reasoning models, including chain-of-thought monitoring and tests of whether the model would deceive evaluators when its goals conflicted with its training [8].
Anthropic publishes detailed model cards and system cards for its Claude family. The Claude 3 model card (March 2024) documented constitutional AI training, included multimodal Trust & Safety red-teaming on topics such as child safety, dangerous weapons, hate speech, violent extremism, and fraud, along with sections on elections integrity, discrimination, and biological, cyber, and autonomous-replication evaluations. Anthropic followed it with addenda and then standalone system cards for Claude 3.5 Sonnet (June 2024), the upgraded Claude 3.5 Sonnet and Claude 3.5 Haiku (October 2024), Claude 3.7 Sonnet, Claude Opus 4 and Sonnet 4 (May 2025), Claude Opus 4.5 (November 2025), and later releases through Claude Opus 5.5 and Sonnet 5.5 (September 2026) and Claude Haiku 5.5 (October 2026). Each release ties to Anthropic's Responsible Scaling Policy, which mandates specific evaluations in CBRN (Chemical, Biological, Radiological, Nuclear), cybersecurity, and autonomous capability domains before deployment [9][10].
Google DeepMind has published technical reports that serve a comparable function for Gemini 1.0, 1.5, 2.0, and later releases, while Meta has published model cards for its Llama series. The DeepSeek-V3 Technical Report (December 2024) and DeepSeek R1 Technical Report (January 2025) similarly act as system cards for those models, covering architecture (Mixture-of-Experts with 671B total / 37B active parameters for V3), training data scale (14.8 trillion tokens), training compute (2.788M H800 GPU hours), and downstream evaluations [11]. The common thread is that as AI systems become more complex and capable, documentation needs to cover not just the model itself but the full stack of engineering, safety, and deployment decisions around it.
Model card versus system card
| Aspect | Model card | System card |
|---|---|---|
| Scope | Single trained model | Entire deployed system including model, tooling, mitigations |
| Typical length | 1 to 10 pages | 30 to 200+ pages for frontier LLMs |
| Safety content | Known limitations | Red-team results, dangerous-capability evals, mitigation efficacy |
| Audience | ML practitioners, downstream developers | Developers, regulators, policymakers, civil society |
| Update cadence | Often static after release | Often updated with new evaluations or mitigations |
| Examples | Llama 3 model card, Mistral 7B model card | GPT-4 System Card, Claude Opus 4.5 System Card, Operator System Card |
What tools help create model cards?
A growing ecosystem of open-source tools makes model card creation easier. Each addresses a slightly different audience, from individual researchers to enterprise governance teams.
| Tool | Maintainer | Purpose |
|---|---|---|
| Model Card Toolkit (MCT) | TensorFlow / Google | Python library that auto-populates JSON schema from ML Metadata, integrates with TensorFlow Extended (TFX), renders to HTML; repository archived September 2024 |
| Hugging Face Model Card Creator | Hugging Face | Web UI for filling in the standardized template without writing markdown |
| huggingface_hub library | Hugging Face | Python ModelCard and ModelCardData classes for programmatic creation, validation, and pushing of cards |
| AI Factsheets | IBM (watsonx.governance) | Lifecycle tracking from training to production, integrated with model inventory; based on Arnold et al. "FactSheets" research |
| VerifyML | Cylynx | Open-source auditing tool that pairs with model cards to verify reported performance |
The Model Card Toolkit (MCT), open-sourced in July 2020 at github.com/tensorflow/model-card-toolkit, was Google's first public attempt to make model card production routine for engineers. It provides a JSON schema, a Python API, and a default HTML template, and can pull model lineage out of ML Metadata so that fields like training dataset and evaluation metrics populate automatically [22]. IBM's AI FactSheets, building on "FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity" (Arnold et al., IBM Journal of Research and Development), takes a more enterprise-governance approach, treating each model as an asset in a tracked inventory [21].
How does regulation use model cards?
Documentation requirements for AI systems have moved from voluntary best practice to legal mandate in several jurisdictions, particularly between 2023 and 2026. The regulations rarely use the term "model card" explicitly, but the information they require maps closely to the Mitchell et al. framework.
EU AI Act
The EU AI Act, formally Regulation (EU) 2024/1689, was published in the Official Journal on July 12, 2024 and entered into force on August 1, 2024. It includes specific documentation requirements for AI systems, particularly those classified as high-risk. Providers of high-risk AI systems must maintain comprehensive technical documentation covering the system's intended purpose, design, development methodology, training and testing data, performance metrics, risk management measures, and post-market monitoring plans [12].
For general-purpose AI (GPAI) models, including LLMs, the AI Act requires providers to publish a sufficiently detailed summary of the training data content, using a template that the EU's AI Office published on July 24, 2025. They must also draw up technical documentation covering the training and testing process and evaluation results, share documentation with downstream providers integrating the model, maintain that documentation per model version for ten years, and comply with EU copyright law. GPAI obligations entered into application on August 2, 2025, with full AI Office enforcement powers from August 2, 2026; providers of models placed on the market before August 2, 2025 have until August 2, 2027 to comply [12][13].
Providers of GPAI models with systemic risk (those exceeding 10^25 floating-point operations in training, a threshold designed to capture frontier models like GPT-4 and Claude Opus) face additional obligations: model evaluations including adversarial testing, serious incident reporting, cybersecurity protections, and assessment and mitigation of systemic risks. Free and open-source models receive partial exemptions from the technical documentation and downstream support obligations but still must publish a training data summary and respect copyright [12].
The AI Act's documentation requirements effectively mandate something resembling model cards for high-risk AI systems and general-purpose AI models placed on the EU market. While the regulation does not specifically require the Mitchell et al. format, the overlap between regulatory requirements and model card best practices has led many organizations to use model cards as a starting point for compliance.
NIST AI Risk Management Framework
The US NIST AI RMF (AI 100-1), published in January 2023, organizes responsible AI practice around four core functions: GOVERN, MAP, MEASURE, and MANAGE. Documentation runs through all four. The MAP function calls for documenting context, risks, and intended uses; MEASURE calls for documenting evaluation methods and results; GOVERN calls for organizational policies that make documentation systematic; MANAGE calls for documenting risk responses and monitoring. The framework explicitly treats documentation as enabling transparency, accountability, and human review, even though it does not prescribe a specific template [14].
NIST has also published the Generative AI Profile (NIST AI 600-1, July 2024) that adapts the RMF specifically to generative models, with documentation expectations tailored to risks like data leakage, hallucination, and content provenance.
United States executive orders
In October 2023 the Biden administration issued Executive Order 14110, "Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence," which required developers of dual-use foundation models above specified compute thresholds to share safety test results with the federal government and contemplated reporting requirements that mirrored model card content. The order also directed NIST to develop a generative-AI-focused resource to supplement the AI Risk Management Framework [15].
On January 20, 2025, President Trump rescinded EO 14110 via Executive Order 14148. Three days later, Trump signed "Removing Barriers to American Leadership in Artificial Intelligence," which directs federal agencies to draft an AI Action Plan within 180 days. As of late 2026, federal documentation requirements for foundation models in the United States remain unsettled. State laws fill part of the gap, notably California's SB 53, the Transparency in Frontier Artificial Intelligence Act (signed September 29, 2025), which requires large frontier developers to publish a frontier AI framework and creates a channel for reporting critical safety incidents [15][16][28].
Colorado AI Act
Colorado's Senate Bill 24-205, signed May 17, 2024 and known as the Colorado Anti-Discrimination in AI Law (ADAI), was the first state-level high-risk AI statute in the United States. It distinguishes between developers (who build or substantially modify high-risk AI systems) and deployers (who put them into use). Developers must give deployers a statement describing reasonably foreseeable uses, known harmful or inappropriate uses, the type of data used to train the system, and known or foreseeable limitations. They must also disclose any known risk of algorithmic discrimination to the Colorado Attorney General and to all known deployers within 90 days of discovery. Deployers must run impact assessments, maintain organized records (impact assessments, testing results, data source descriptions, version histories, internal review approvals), and notify consumers when a high-risk system is used to make a consequential decision about them. Those provisions were due to take effect June 30, 2026, but on May 14, 2026 Governor Jared Polis signed SB 26-189, which repealed and reenacted the law before it took effect. The replacement covers automated decision-making technology used in consequential decisions: from January 1, 2027, developers must give deployers technical documentation on intended uses, categories of training data, known limitations, and instructions for appropriate use and human review, deployers must notify consumers, and both must keep compliance records for at least three years [17][27].
Regulatory landscape at a glance
| Framework | Jurisdiction | In force | Documentation core |
|---|---|---|---|
| EU AI Act | European Union | Aug 1, 2024 (GPAI obligations Aug 2, 2025) | Technical documentation, training data summary, downstream provider info |
| NIST AI RMF 1.0 | United States (voluntary) | Jan 26, 2023 | GOVERN/MAP/MEASURE/MANAGE functions, transparency woven through |
| NIST AI 600-1 (Gen AI Profile) | United States (voluntary) | Jul 26, 2024 | Generative-AI-specific risks and documentation |
| EO 14110 (Biden) | United States | Oct 30, 2023 (rescinded Jan 20, 2025) | Safety test reporting for dual-use models above compute thresholds |
| EO 14179 (Trump) | United States | Jan 23, 2025 | Action plan; rolls back prior reporting requirements |
| Colorado SB 24-205 (ADAI), replaced by SB 26-189 | Colorado, US | SB 26-189 effective Jan 1, 2027 (SB 24-205 repealed before taking effect) | Developer technical documentation for deployers, consumer notice, 3-year record retention |
| Singapore Model AI Governance Framework | Singapore (voluntary) | 2019 (rev. 2020, 2024) | Documentation of design, data, and decisions |
| Canada Algorithmic Impact Assessment | Canada (federal use) | 2020 | Mandatory questionnaire for federal AI systems |
What do real model cards look like?
The following table lists notable model cards and system cards published by major AI organizations, illustrating the range of documentation practices in use. Tables in this article are exempt from the one-link-per-article rule, so the same wikilinks may appear across rows.
| Organization | Model | Document type | Year | Notable features |
|---|---|---|---|---|
| Smile detector, Toxicity (Perspective) | Model card | 2019 | Original examples from the Mitchell et al. paper; revealed disparate performance for older men and for sexual-orientation terms | |
| Gemma | Model card | 2024 | Open-weights companion to Gemini; standardized template across Gemma 1 and 2 | |
| Google DeepMind | Gemini 1.0, 1.5, 2.0 | Technical report | 2023, 2024 | De facto system card; long-context evaluations and red-team results |
| OpenAI | GPT-4 | System card | March 2023 | Red-team results across CBRN, persuasion, cybersecurity; more than 50 external expert red-teamers |
| OpenAI | GPT-4V (vision) | System card | September 2023 | Image-input safety, person identification, biometric inference |
| OpenAI | GPT-4o | System card | August 2024 | Multimodal (text/image/audio); voice safety, speaker ID, ungrounded inference |
| OpenAI | o1 | System card | December 2024 | First detailed evaluation of chain-of-thought reasoning safety |
| OpenAI | Operator | System card | 2025 | Agentic browser use; documentation of containment and oversight |
| Anthropic | Claude 3 (Opus, Sonnet, Haiku) | Model card | March 2024 | Constitutional AI training, multimodal Trust & Safety red-teaming, elections integrity |
| Anthropic | Claude 3.5 Sonnet (and addendum) | Model card | June 2024, Oct 2024 | Document analysis, visual understanding, coding evaluations |
| Anthropic | Claude 3.7 Sonnet | System card | 2025 | First Anthropic system card under updated RSP |
| Anthropic | Claude Opus 4 / Sonnet 4 | System card | May 2025 | Hybrid-reasoning models; CBRN and autonomy evaluations |
| Anthropic | Claude Opus 4.5 | System card | November 2025 | Released under the RSP's ASL-3 Standard; 153 pages |
| Meta | Llama 2 | Model card | July 2023 | Detailed benchmarks and Responsible Use Guide; published on Hugging Face |
| Meta | Llama 3 (8B, 70B) | Model card | April 2024 | 128K-token tokenizer, 8,192-token sequences; CyberSecEval and CBRNE testing |
| Meta | Llama 3.1 / 3.2 / 3.3 | Model card | 2024 | Multilingual coverage, 405B variant, multimodal extensions |
| Mistral AI | Mistral 7B, Mixtral 8x7B | Model card | 2023 | Concise template focused on benchmarks and Apache 2.0 license |
| Stability AI | Stable Diffusion | Model card | 2022 | Documented training data sources, biases in image generation, misuse risks |
| DeepSeek | DeepSeek-V3 | Technical report | December 2024 | 671B MoE with 37B active; 14.8T training tokens; 2.788M H800 GPU hours |
| DeepSeek | DeepSeek-R1 | Technical report | January 2025 | Reasoning model trained with large-scale reinforcement learning after a cold-start data stage (the R1-Zero variant used RL only) |
The variation between these documents reflects both organizational priorities and the absence of a binding standard. OpenAI and Anthropic have converged on long, multi-section system cards (often 50 to 200+ pages for frontier models; the Claude Opus 4.5 system card alone runs 153 pages). Meta and Mistral favor compact model cards. Open-source community releases sit anywhere on the spectrum.
What sections appear in modern LLM model cards?
For large language models, the standard model card template has expanded well beyond the original Mitchell et al. nine sections. A typical 2024 to 2026 LLM model card includes:
| Section | Typical content |
|---|---|
| Architecture | Model family, decoder-only or MoE, parameter count, active parameters, attention type (e.g., grouped-query, multi-head latent), context length |
| Training data | Composition (web, code, books, math, multilingual), token count, cutoff date, deduplication and filtering |
| Training compute | FLOPs, hardware (e.g., H100, H800, TPU v5p), GPU-hours, training duration |
| Energy and carbon | kWh consumed, location-based and market-based CO2e, sometimes water usage |
| Capabilities benchmarks | MMLU, GPQA, HumanEval, MATH, BIG-Bench Hard, MMMU, AIME, SWE-bench, GDPval |
| Long-context evaluations | Needle-in-a-haystack, RULER, multi-document QA |
| Safety evaluations | Red-team results, refusal rates, jailbreak resistance, sycophancy and deception checks |
| Bias and fairness | StereoSet, BBQ, CrowS-Pairs, multilingual fairness checks |
| Alignment evaluations | Constitutional AI compliance, instruction-following, RLHF win rates |
| Dangerous-capability evals | CBRN uplift, cybersecurity (CTF, vulnerability discovery), autonomous replication |
| Suggested uses | Recommended applications and audiences |
| Out-of-scope uses | Applications the developer disclaims (e.g., medical diagnosis, legal advice) |
| License and access | Apache 2.0, MIT, Llama Community License, custom RAIL license, API-only access |
Not every model card hits every section. Open-weights releases are more likely to publish carbon and compute numbers; closed-weights frontier releases tend to invest more in safety and dangerous-capability sections.
What are the limitations and criticisms of model cards?
Despite their wide adoption, model cards have been subject to several substantive critiques. Many of these limitations are acknowledged in the original Mitchell et al. paper, but they have not been fully resolved by subsequent practice.
Voluntary and inconsistent. Outside of regulated environments, model card creation is voluntary. Many models, particularly those released by smaller organizations or individual researchers, ship with incomplete or nonexistent documentation. The Liang et al. (2024) systematic analysis of 32,111 Hub model cards found that even the average card had multiple sections missing or stub-quality, with environmental, limitations, and evaluation sections least likely to be filled in [5].
Static documents. Model cards are typically created at the time of model release and rarely updated thereafter. As models are fine-tuned, deployed in new contexts, or discovered to have previously unknown failure modes, the original model card becomes outdated. Some organizations have adopted versioned model cards (Anthropic publishes addenda for each Claude release; Meta updates Llama cards across 3.1, 3.2, 3.3), but this practice is not widespread. Reward Reports (Gilbert et al., 2023) were proposed specifically to address the dynamic-system blind spot, focusing on reinforcement learning systems whose objectives evolve with deployment [18].
Self-reported. Model cards are written by the organizations that develop and release the models. There is an inherent tension between transparency and self-interest: organizations may understate risks, overstate performance, or omit information about known problems. Independent auditing of model card claims remains rare. Inioluwa Deborah Raji and colleagues have proposed an end-to-end internal algorithmic auditing framework (SMACTR) in which each stage of development yields audit documents [19].
Documentation theater. Critics have warned that producing a model card can substitute for actually doing the underlying safety, fairness, or accountability work. A polished card with the right section headers can give the impression of due diligence while the model itself was never tested in the relevant ways. This concern parallels longer-standing critiques of "ethics washing" in tech.
Readability and audience. Model cards are often written in technical language that is accessible to ML practitioners but opaque to policymakers, journalists, affected communities, and other stakeholders who may need the information most. The original Mitchell et al. paper framed model cards as serving many stakeholders, from ML practitioners to policymakers and regulators; in practice, most cards still read like research-paper appendices.
Scope limitations. A model card documents a model; it does not document the deployment context, the downstream applications, or the lived experiences of people affected by the model's outputs. A toxicity classifier's model card might document that the model performs well on a standard benchmark, without capturing how its deployment in a content moderation system affects free expression for specific communities. System cards partially address this gap, but deployment-level documentation remains underdeveloped.
Benchmark gaming. As model cards have become competitive marketing artifacts, the benchmarks they report have become subject to contamination, cherry-picking, and tuning-for-the-test. The cards themselves are not at fault, but they have helped concentrate attention on a narrow set of metrics that may not generalize.
How does the broader ML documentation landscape fit together?
Model cards exist within a broader ecosystem of documentation and governance tools for machine learning. Hugging Face maintains a Landscape of ML Documentation Tools resource that catalogs these complementary frameworks; each addresses a different layer of the AI lifecycle.
| Tool / Framework | Purpose | Originating work |
|---|---|---|
| Model cards | Document individual ML models | Mitchell et al., FAT* 2019 |
| Datasheets for datasets | Document training and evaluation datasets | Gebru et al., CACM 2021 |
| System cards | Document deployed AI systems including safety measures | Meta AI (2022); popularized by OpenAI (GPT-4 System Card, 2023) |
| AI FactSheets | Comprehensive AI governance documentation | Arnold et al., IBM 2019 |
| Data Statements | Document characteristics of NLP datasets | Bender and Friedman, TACL 2018 (v3 2023) |
| Reward Reports | Document objectives and reward functions of reinforcement learning systems | Gilbert et al., AIES 2023 |
| Dataset Nutrition Labels | Standardized nutrition-label-style data summary | Holland et al., 2018 |
| Responsible AI Licenses (RAIL) | License terms with use restrictions for AI models | RAIL Initiative |
| Algorithmic Impact Assessments | Pre-deployment risk assessment of AI systems | Government of Canada, 2020 |
These tools serve overlapping but distinct purposes. No single document can capture all relevant information about an AI system, from raw training data to real-world impact. The trend in AI documentation is toward complementary, layered documentation that covers different aspects of the AI lifecycle, with model cards anchoring the model layer while datasheets cover data, system cards cover deployment, and reward reports cover dynamic behavior.
What is the current state and outlook for model cards?
As of 2026, model cards are firmly established as a norm in the AI industry, though their adoption and quality remain uneven. The EU AI Act's GPAI obligations have been in force since August 2025, and the AI Office's enforcement powers took effect on August 2, 2026, which is pulling the median quality of frontier-model documentation upward. Providers selling into the European market increasingly publish documentation that satisfies EU technical-documentation expectations regardless of where the company itself sits.
Hugging Face continues to lead in model card tooling and infrastructure. The platform now hosts more than two million public models, with automated card generation features that pre-populate certain fields based on metadata, training logs, and benchmark results [24]. Research groups have explored using language models themselves to draft model card content, an idea that raises awkward questions about accuracy and accountability when the documenter is the same kind of system being documented.
Frontier-lab system cards have grown dramatically in scope. Where the GPT-4 System Card in 2023 ran roughly 60 pages, more recent releases such as the 153-page Claude Opus 4.5 system card run well over 100 pages, with detailed sections on dangerous-capability evaluations (CBRN uplift studies, cyber CTF performance, autonomous replication tests), agentic-task evaluations like SWE-bench and OpenAI's GDPval gold dataset of 220 tasks across 44 occupations, and a growing focus on evaluation reproducibility. Anthropic's Responsible Scaling Policy, OpenAI's Preparedness Framework, and Google DeepMind's Frontier Safety Framework all tie release decisions to specific evaluations that get reported back in the system card.
The broader trajectory points toward AI documentation becoming not just a best practice but a legal and commercial requirement. As AI systems are deployed in healthcare, finance, criminal justice, and defense, the demand for thorough, accurate, and independently verifiable documentation will only grow. Model cards, for all their limitations, established the vocabulary and expectations that more rigorous successors are now being built on.
Continue reading
- Hugging FaceHugging Face is an artificial intelligence company and collaboration platform for machine learning.Mentioned 19 times above
- BiasBias in artificial intelligence carries three distinct technical meanings: a learnable scalar parameter added inside a neuron, the systematic error component of an estimator (the "bias" in the bias-variance…Mentioned 6 times above
- EU AI ActThe EU AI Act is the common name for Regulation (EU) 2024/1689, a directly applicable European Union law that sets horizontal rules for artificial intelligence.Mentioned 5 times above
- Timnit GebruTimnit Gebru is an Ethiopian-born computer scientist and a leading researcher in AI ethics, best known for co-authoring the 2018 "Gender Shades" study on bias in facial recognition, co-leading Google's Ethical…Mentioned 4 times above
- Responsible AIResponsible AI (RAI) is a framework for developing, deploying, and governing artificial intelligence systems in ways that are ethical, transparent, accountable, and aligned with human values.Mentioned 3 times above
- AI governanceAI governance is the collection of frameworks, norms, standards, policies, and institutional arrangements that guide the development, deployment, and use of artificial intelligence systems so that they are…Mentioned 3 times above
More from See also
- Datasheet
- AI transparency
- AI ethics
- Explainable AI
- AI safety
- AI Alignment
- NIST AI Risk Management Framework
- Margaret Mitchell (computer scientist)
- Red teaming (artificial intelligence)
- Responsible Scaling Policy
Topics:AI EthicsDeveloper Tools
References
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I.D., and Gebru, T. "Model Cards for Model Reporting." Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* 2019), 220-229. arxiv.org/...1810.03993 and dl.acm.org/...3287560.3287596
- ^Mitchell, M. Personal site and biography. m-mitchell.com ; "Margaret Mitchell (scientist)." Wikipedia. en.wikipedia.org/...Margaret_Mitchell_(scientist)
- ^1 ^2 ^3 ^4Hugging Face. "Model Cards." huggingface.co/...model-cards ; Hugging Face. "Model Card Guidebook." huggingface.co/...model-card-guidebook ; Hugging Face. "Annotated Model Card Template." huggingface.co/...model-card-annotated
- ^Hugging Face. "huggingface_hub v1.0: Five Years of Building the Foundation of Open Machine Learning." Blog post, 2025. huggingface.co/...huggingface-hub-v1
- ^1 ^2Liang, W., Rajani, N., Yang, X., Ozoani, E., Wu, E., Chen, Y., Smith, D.S., and Zou, J. "Systematic analysis of 32,111 AI model cards characterizes documentation practice in AI." Nature Machine Intelligence, 2024. nature.com/...s42256-024-00857-z ; preprint "What's documented in AI? Systematic Analysis of 32K AI Model Cards." arxiv.org/...2402.05160
- ^1 ^2 ^3Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wallach, H., Daume III, H., and Crawford, K. "Datasheets for Datasets." Communications of the ACM, Vol. 64, No. 12, December 2021, 86-92. arxiv.org/...1803.09010 and dl.acm.org/...3458723
- ^Dickey, M.R. "Google fires top AI ethics researcher Margaret Mitchell." TechCrunch, February 19, 2021. techcrunch.com/...ics-researcher-margaret-mitchell
- ^1 ^2 ^3 ^4OpenAI. "GPT-4 System Card." March 23, 2023. cdn.openai.com/...gpt-4-system-card.pdf ; OpenAI. "GPT-4o System Card." August 2024. openai.com/...gpt-4o-system-card ; OpenAI. "Operator System Card." 2025. openai.com/...operator-system-card
- ^Anthropic. "The Claude 3 Model Family: Opus, Sonnet, Haiku." Model card, March 2024. www-cdn.anthropic.com/...Model_Card_Claude_3.pdf ; Anthropic. "Model system cards." anthropic.com/system-cards
- ^Anthropic. "Anthropic's Responsible Scaling Policy." Versions 1.0 (Sept 2023), 2.0 (Oct 2024), 3.0 (Feb 2026). anthropic.com/responsible-scaling-policy
- ^DeepSeek-AI. "DeepSeek-V3 Technical Report." arXiv:2412.19437, December 2024. arxiv.org/...2412.19437 ; DeepSeek-AI. "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning." January 2025. huggingface.co/...DeepSeek-R1
- ^1 ^2 ^3European Commission. "AI Act." Regulation (EU) 2024/1689. digital-strategy.ec.europa.eu/...tory-framework-ai ; European Commission. "AI Act enters into force." August 1, 2024. commission.europa.eu/...enters-force-2024-08-01_en ; European Commission. "The General-Purpose AI Code of Practice." digital-strategy.ec.europa.eu/...contents-code-gpai
- ^European Commission. "EU rules on general-purpose AI models start to apply." August 2025. digital-strategy.ec.europa.eu/...nd-accountability ; Future of Life Institute. "High-level summary of the AI Act." artificialintelligenceact.eu/high-level-summary
- ^National Institute of Standards and Technology. "Artificial Intelligence Risk Management Framework (AI RMF 1.0)." NIST AI 100-1, January 2023. nvlpubs.nist.gov/...nist.ai.100-1.pdf ; NIST. "AI Risk Management Framework: Generative AI Profile." NIST AI 600-1, July 2024. nvlpubs.nist.gov/...NIST.AI.600-1.pdf
- ^1 ^2White House. "Executive Order 14110: Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence." October 30, 2023. en.wikipedia.org/...Executive_Order_14110
- ^White House. "Executive Order 14148: Initial Rescissions of Harmful Executive Orders and Actions." January 20, 2025; "Removing Barriers to American Leadership in Artificial Intelligence." January 23, 2025. aila.org/...-leadership-in-artificial-intelligence
- ^Colorado General Assembly. "SB24-205 Consumer Protections for Artificial Intelligence." Signed May 17, 2024, effective June 30, 2026. leg.colorado.gov/...sb24-205
- ^Gilbert, T.K., Lambert, N., Dean, S., Zick, T., and Snoswell, A. "Reward Reports for Reinforcement Learning." Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. arxiv.org/...2204.10817
- ^Raji, I.D., Smart, A., White, R.N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., and Barnes, P. "Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing." FAT* 2020. dl.acm.org/...3351095.3372873
- Bender, E.M. and Friedman, B. "Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science." Transactions of the Association for Computational Linguistics, 2018. aclanthology.org/Q18-1041
- ^Arnold, M., Bellamy, R.K.E., Hind, M., et al. "FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity." IBM Journal of Research and Development. research.ibm.com/...aifactsheets
- ^TensorFlow / Google. "Model Card Toolkit." github.com/...model-card-toolkit ; "Introducing the Model Card Toolkit for Easier Model Transparency Reporting." Google Research blog, July 2020. research.google/...er-model-transparency-reporting
- Meta. "Llama 3 Model Card." April 2024. github.com/...MODEL_CARD.md ; "Llama 3.1 Model Card." github.com/...MODEL_CARD.md
- ^1 ^2 ^3Hugging Face. "huggingface_hub v1.0: Five Years of Building the Foundation of Open Machine Learning." 2025. huggingface.co/...huggingface-hub-v1 ; Laufer, B., Oderinwale, H., and Kleinberg, J. "Anatomy of a Machine Learning Ecosystem: 2 Million Models on Hugging Face." arXiv:2508.06811, 2025. arxiv.org/...2508.06811
- ^Rios, F. "Hugging Face's two million models and counting." AI World (CEPS), December 8, 2025. aiworld.eu/...aces-two-million-models-and-counting
- ^Mind-Verse. "Hugging Face Hub: Meilenstein von 3 Millionen KI-Modellen." 2026. mind-verse.de/...eilenstein-3-millionen-ki-modelle
- ^Colorado General Assembly. "SB26-189 Automated Decision-Making Technology." Signed May 14, 2026. leg.colorado.gov/...sb26-189
- ^Office of the Governor of California. "Governor Newsom signs SB 53, advancing California's world-leading artificial intelligence industry." September 29, 2025. gov.ca.gov/...ing-artificial-intelligence-industry
Improve this article
Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.
8 revisions · v8 · 7,071 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify
Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here
Reviewer note: Independent fact-check of the full article against its cited sources (150 claims, 44 sources fetched); 12 material + 23 minor fixed
Cite this page: AI Wiki. "Model card." aiwiki.ai, updated 23 Jun 2026, fact-checked 1 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/model_card


