Model card

34 min read
Updated
Suggest editHistoryTalk
RawGraph

Last edited

Fact-checked

In review queue

Sources

24 citations

Revision

v5 · 6,810 words

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

A model card is a short, standardized document that accompanies a trained machine learning model and reports its intended use, training data, evaluation results across different population subgroups, ethical considerations, and known limitations. Model cards were introduced by Margaret Mitchell, Timnit Gebru, and colleagues at Google in the 2019 paper "Model Cards for Model Reporting," which defines them as "short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups" [1]. They function as a form of AI transparency documentation, analogous to nutrition labels for food, datasheets for electronic components, or Material Safety Data Sheets in chemistry, and have become a de facto industry standard largely through Hugging Face, where every one of the platform's 2 million-plus public models carries a model card as its README file [1][3][24].

Model cards are part of a broader ecosystem of AI documentation practices that includes datasheets for datasets (Gebru et al., 2021), system cards (used by OpenAI for models like GPT-4), and regulatory documentation requirements under frameworks like the EU AI Act. Together, these documentation standards aim to improve accountability, reproducibility, and informed decision-making across the AI development lifecycle. They sit at the intersection of responsible AI, AI governance, and AI safety practice.

Who created model cards, and when?

The foundational paper, "Model Cards for Model Reporting," was published at the ACM Conference on Fairness, Accountability, and Transparency (FAT*, later renamed FAccT), held January 29-31, 2019 in Atlanta. The paper was authored by Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru, all of whom were affiliated with Google at the time. The first preprint was posted to arXiv (1810.03993) on October 5, 2018, with the revised version finalized January 14, 2019 [1].

Mitchell co-founded and co-led Google's Ethical AI team alongside Gebru. The model cards proposal grew out of that team's broader work on fairness, accountability, and bias in deployed machine learning. After Gebru's controversial departure from Google in December 2020, Mitchell was also fired in February 2021 following an internal investigation. She joined Hugging Face later that year as Chief Ethics Scientist, where she continued the model cards work and helped shape the platform's documentation practices [2][7].

The paper identified a gap in how machine learning models were documented and shared. While software engineering had established practices for documentation (API references, changelogs, README files), machine learning models were frequently released with minimal information about their intended use, performance across different populations, training data composition, or known limitations. This lack of transparency made it difficult for downstream users to evaluate whether a model was appropriate for their specific application, and it obscured potential harms to affected communities [1].

Mitchell et al. proposed model cards as a short, structured document accompanying any trained machine learning model. The paper drew inspiration from existing practices in other fields: datasheets in the electronics industry, nutritional labels in the food industry, and Material Safety Data Sheets in chemistry. The authors argued that a standardized documentation format would facilitate better communication between model developers and model users, encourage the evaluation of models across different demographic groups, and create accountability for known limitations [1].

The paper's most cited definition reads: "Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains." The emphasis on intersectional evaluation, drawing on critical race theory and earlier fairness work, was a deliberate departure from aggregate accuracy reporting [1].

The paper demonstrated the concept with example model cards for two Google models: a smile detector trained on the CelebA dataset and a toxicity classifier built on the Perspective API. The smile-detection card revealed higher false discovery rates for older men. The toxicity card surfaced bias against terms related to sexual orientation. In both cases the disaggregated reporting exposed disparities that aggregate accuracy numbers had hidden, demonstrating exactly the failure mode the authors were trying to address [1].

What does a model card contain?

The original Mitchell et al. framework specifies nine sections that a model card should include. These sections have been refined and expanded by subsequent adopters, but the core structure remains consistent across most modern templates [1].

SectionDescriptionExample content
Model detailsBasic information about the modelName, version, type (e.g., classification, generation), developer, release date, license, citation, contact
Intended useWhat the model was designed forPrimary use cases, intended users, out-of-scope uses
FactorsVariables that affect model performanceDemographic factors (age, gender, ethnicity, Fitzpatrick skin type), environmental factors (lighting, noise), instrumentation factors (camera type, microphone quality)
MetricsHow performance is measuredAccuracy, F1 score, BLEU, perplexity, fairness metrics, decision thresholds, variation approaches, disaggregated by relevant factors
Evaluation dataDatasets used for testingDataset name, size, composition, motivation for choice, any known biases
Training dataDatasets used for trainingSources, size, collection methodology, preprocessing steps, any known limitations
Quantitative analysesDetailed performance resultsUnitary results per factor, intersectional results, confidence intervals, error analysis
Ethical considerationsKnown risks and societal impactPotential for harm, sensitive use cases, populations at risk, mitigation strategies considered during development
Caveats and recommendationsKnown limitations and usage guidanceFailure modes, conditions under which the model should not be used, recommended downstream evaluation

The emphasis on disaggregated evaluation, where performance metrics are broken down across demographic or contextual subgroups rather than reported only as aggregate numbers, is one of the most important contributions of the model card framework. A face recognition model might achieve 99% accuracy overall but only 85% accuracy on darker-skinned faces; an aggregate metric would hide this disparity, while a model card would make it explicit. This approach was directly informed by Joy Buolamwini and Timnit Gebru's earlier Gender Shades research, which found commercial face analysis systems performed dramatically worse on darker-skinned women than on lighter-skinned men [1].

The paper also proposed two specific reporting modes: unitary results, which show performance per single factor, and intersectional results, which show performance at the intersection of factors (for example age x skin type). The intersectional view is more demanding but typically reveals worse outcomes than either factor alone would suggest, which is precisely why Mitchell et al. argued it was needed.

How does Hugging Face implement model cards?

Hugging Face, the open-source AI platform and model hosting service, has become the primary venue where model cards are created and consumed in practice. The Hugging Face Hub crossed one million hosted models in 2024 and surpassed two million in 2025, reaching roughly 2.4 million public models by 2026, with each model repository containing a README.md file that serves as the model card [3][4][24]. The first million models took more than 1,000 days to accumulate after the Hub began tracking in March 2022, while the second million arrived just 335 days later, a roughly 65% reduction in the time required, driven largely by a wave of small fine-tuned models [24].

Hugging Face's adoption of model cards has been the single most significant factor in their widespread use. When a researcher or organization uploads a model to the Hub, the platform prompts them to fill out a model card using a standardized template. This template draws on the original Mitchell et al. framework but has been expanded to include additional metadata fields relevant to the Hugging Face ecosystem, such as pipeline tags (text-classification, image-generation, etc.), language codes, datasets used, and library compatibility [3].

In 2022, Hugging Face launched the Model Card Guidebook, authored primarily by Ezi Ozoani, Marissa Gerchick, and Margaret Mitchell. The guidebook bundled four resources: an Annotated Model Card Template explaining how to fill out each section, a user study on model card usage at the Hub, a landscape analysis of the broader documentation ecosystem, and a Model Card Creator Tool that allows users to generate model cards through a graphical interface without writing markdown directly. The release also added prompt text inside the template to encourage thorough completion of the Bias, Risks and Limitations section [3].

The quality of model cards on Hugging Face varies significantly. Models released by major organizations (Google, Meta, Microsoft, Anthropic) typically have detailed, well-structured model cards. Community-contributed models, which make up the majority of Hub content, range from thorough documentation to completely empty README files. A 2024 analysis published in Nature Machine Intelligence by Liang et al. systematically examined 32,111 AI model cards on the Hub (drawn from 74,970 models). The study found that the Training section was the most consistently filled-out, while the Environmental Impact, Limitations, and Evaluation sections had the lowest completion rates. The same paper showed that improving model card completeness correlated with higher download rates, suggesting documentation pays off in adoption as well as accountability [5].

What metadata goes in a Hugging Face model card?

Hugging Face model cards include a YAML metadata block at the top of the README.md file, delimited by triple-dash lines. This block enables structured search, filtering, and integration with the platform's API. Key metadata fields include:

FieldPurpose
languageISO 639-1 codes for languages the model supports or was trained on
licenseLicense identifier (apache-2.0, mit, llama3, etc.)
tagsFree-form tags for categorization and discovery
datasetsHub datasets used for training or evaluation
metricsPerformance metrics and their reported values
pipeline_tagThe task type (text-generation, image-classification, automatic-speech-recognition, etc.)
library_nameThe compatible ML library (transformers, diffusers, sentence-transformers); required for transformers repos created after August 2024
base_modelThe model this one was fine-tuned, adapted, or quantized from
thumbnailImage used in social media previews
model-indexStructured evaluation results with task, dataset, metric, and source fields

The model-index field deserves special mention because it lets the Hub surface evaluation numbers in a standardized way. A repo can declare, for example, that the model achieved 87.3 on MMLU using the lm-eval-harness, and that result will appear on aggregated leaderboards rather than buried in prose.

How do datasheets for datasets relate to model cards?

Model cards were designed to document models, but a parallel documentation framework exists for the datasets used to train and evaluate those models. "Datasheets for Datasets," proposed by Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daume III, and Kate Crawford, was first published as a preprint in March 2018 and later appeared in Communications of the ACM in December 2021 [6].

The datasheets framework takes its name from the electronics industry, where every component ships with a datasheet specifying its operating characteristics. Gebru et al. argued that every dataset used in machine learning should be accompanied by a similar document covering motivation, composition, the data collection process, any preprocessing or labeling steps, recommended uses, distribution, maintenance plans, and legal or ethical considerations. The full questionnaire contains 57 questions across seven categories, designed to be answered before a dataset is shared widely [6].

The key questions a dataset datasheet answers include:

CategoryExample questions
MotivationWhy was the dataset created? Who created it? Who funded it?
CompositionWhat do the instances represent? How many are there? Is there missing data? Does it contain confidential information?
Collection processHow was the data collected? Who was involved? Over what timeframe? Were individuals notified? Did they consent?
PreprocessingWas any data cleaned, filtered, or labeled? What tools or procedures were used?
UsesWhat tasks has the dataset been used for? Are there tasks it should not be used for?
DistributionHow is the dataset distributed? Under what license? Are there access restrictions?
MaintenanceWho maintains the dataset? How can errors be reported? Will it be updated?

Datasheets for datasets and model cards are complementary: a model card references the training and evaluation data, while the dataset's datasheet provides the detailed provenance information for that data. Together, they create a documentation chain from raw data to deployed model. Hugging Face later integrated dataset cards (a YAML-fronted README on every dataset repo) that mirror this structure for the Hub [6].

Model cards versus datasheets at a glance

DimensionModel cardDatasheet for datasets
What it documentsA trained machine learning modelA dataset used for training or evaluation
Originating paperMitchell et al., 2019 (FAT*)Gebru et al., 2018 (preprint), 2021 (CACM)
Core sections9 (Model details to Caveats)7 categories, 57 questions
Disaggregation focusPerformance across demographic groupsComposition and collection across population subgroups
Typical authorModel developerDataset curator or maintainer
Common venueHugging Face README, internal toolingDataset README, paper appendix

How does a system card differ from a model card?

System cards represent an evolution of the model card concept, designed for AI systems that are more complex than a single model. The term was popularized by OpenAI, which has published system cards for several of its major releases, including GPT-4 (March 2023), GPT-4V (the vision-enabled variant, September 2023), GPT-4o (August 2024), the o1 reasoning models (December 2024), and subsequent frontier releases [8].

A system card differs from a model card in several respects. While a model card documents a single trained model in isolation, a system card documents the entire system that is deployed to users, including the base model, fine-tuning procedures, safety mitigations (such as RLHF training, content filters, and rate limits), deployment configuration, and the results of red-teaming and safety evaluations. System cards also typically include a discussion of the system's capabilities and the risks those capabilities pose in deployment [8].

OpenAI's GPT-4 System Card, published March 23, 2023 alongside the GPT-4 Technical Report, set the template that later system cards have followed. OpenAI assembled an external red team of 41 researchers across cybersecurity, biosecurity, persuasion, fairness, alignment, and other domains, and ran a months-long testing period in which more than 1,000 early users also probed the model. The card documented specific capabilities that raised safety concerns, including the model's ability to provide step-by-step instructions for synthesizing dangerous chemicals (mitigated through refusal training), its potential to identify private individuals when augmented with outside data, and its performance on standardized exams (passing a simulated bar exam in the 90th percentile). The card also described the limitations of the safety measures in place, including known jailbreaking techniques [8].

The GPT-4o System Card (August 2024) extended this to multimodal systems, adding voice safety evaluations covering unauthorized voice generation, speaker identification, disallowed audio content, and ungrounded inferences. The o1 System Card (December 2024) introduced new evaluations specific to reasoning models, including chain-of-thought monitoring and tests of whether the model would deceive evaluators when its goals conflicted with its training [8].

Anthropic publishes detailed model cards and system cards for its Claude family. The Claude 3 model card (March 2024) documented constitutional AI training, covered fourteen Trust & Safety policy areas across six languages, and included evaluations on areas such as elections integrity, child safety, cyber attacks, hate and discrimination, and violent extremism. The model card has been extended through addenda for Claude 3.5 Sonnet (June 2024), the upgraded Claude 3.5 Sonnet and Claude 3.5 Haiku (October 2024), Claude 3.7 Sonnet, Claude Opus 4 and Sonnet 4 (May 2025), and Claude Opus 4.5 (November 2025). Each release ties to Anthropic's Responsible Scaling Policy, which mandates specific evaluations in CBRN (Chemical, Biological, Radiological, Nuclear), cybersecurity, and autonomous capability domains before deployment [9][10].

Google DeepMind has published technical reports that serve a comparable function for Gemini 1.0, 1.5, 2.0, and later releases, while Meta has published model cards for its Llama series. The DeepSeek-V3 Technical Report (December 2024) and DeepSeek R1 Technical Report (January 2025) similarly act as system cards for those models, covering architecture (Mixture-of-Experts with 671B total / 37B active parameters for V3), training data scale (14.8 trillion tokens), training compute (2.788M H800 GPU hours), and downstream evaluations [11]. The common thread is that as AI systems become more complex and capable, documentation needs to cover not just the model itself but the full stack of engineering, safety, and deployment decisions around it.

Model card versus system card

AspectModel cardSystem card
ScopeSingle trained modelEntire deployed system including model, tooling, mitigations
Typical length1 to 10 pages30 to 200+ pages for frontier LLMs
Safety contentKnown limitationsRed-team results, dangerous-capability evals, mitigation efficacy
AudienceML practitioners, downstream developersDevelopers, regulators, policymakers, civil society
Update cadenceOften static after releaseOften updated with new evaluations or mitigations
ExamplesLlama 3 model card, Mistral 7B model cardGPT-4 System Card, Claude Opus 4.5 System Card, Operator System Card

What tools help create model cards?

A growing ecosystem of open-source tools makes model card creation easier. Each addresses a slightly different audience, from individual researchers to enterprise governance teams.

ToolMaintainerPurpose
Model Card Toolkit (MCT)TensorFlow / GooglePython library that auto-populates JSON schema from ML Metadata, integrates with TensorFlow Extended (TFX), renders to HTML
Hugging Face Model Card CreatorHugging FaceWeb UI for filling in the standardized template without writing markdown
huggingface_hub libraryHugging FacePython ModelCard and ModelCardData classes for programmatic creation, validation, and pushing of cards
AI FactsheetsIBM (watsonx.governance)Lifecycle tracking from training to production, integrated with model inventory; based on Arnold et al. "FactSheets" research
VerifyMLCylynxOpen-source auditing tool that pairs with model cards to verify reported performance
Vertex AI Model CardsGoogle CloudManaged model card generation tied to Vertex AI pipelines

The Model Card Toolkit (MCT), open-sourced in July 2020 at github.com/tensorflow/model-card-toolkit, was Google's first public attempt to make model card production routine for engineers. It provides a JSON schema, a Python API, and a default HTML template, and can pull model lineage out of ML Metadata so that fields like training dataset and evaluation metrics populate automatically [22]. IBM's AI FactSheets, building on "FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity" (Arnold et al., IBM Journal of Research and Development), takes a more enterprise-governance approach, treating each model as an asset in a tracked inventory [21].

How does regulation use model cards?

Documentation requirements for AI systems have moved from voluntary best practice to legal mandate in several jurisdictions, particularly between 2023 and 2026. The regulations rarely use the term "model card" explicitly, but the information they require maps closely to the Mitchell et al. framework.

EU AI Act

The EU AI Act, formally Regulation (EU) 2024/1689, was published in the Official Journal on July 12, 2024 and entered into force on August 1, 2024. It includes specific documentation requirements for AI systems, particularly those classified as high-risk. Providers of high-risk AI systems must maintain comprehensive technical documentation covering the system's intended purpose, design, development methodology, training and testing data, performance metrics, risk management measures, and post-market monitoring plans [12].

For general-purpose AI (GPAI) models, including LLMs, the AI Act requires providers to publish a sufficiently detailed summary of the training data content, using a template that the EU's AI Office published on July 24, 2025. They must also draw up technical documentation covering the training and testing process and evaluation results, share documentation with downstream providers integrating the model, maintain that documentation per model version for ten years, and comply with EU copyright law. GPAI obligations entered into application on August 2, 2025, with full AI Office enforcement powers from August 2, 2026; providers of models placed on the market before August 2, 2025 have until August 2, 2027 to comply [12][13].

Providers of GPAI models with systemic risk (those exceeding 10^25 floating-point operations in training, a threshold designed to capture frontier models like GPT-4 and Claude Opus) face additional obligations: model evaluations including adversarial testing, serious incident reporting, cybersecurity protections, and documentation of energy consumption. Free and open-source models receive partial exemptions from the technical documentation and downstream support obligations but still must publish a training data summary and respect copyright [12].

The AI Act's documentation requirements effectively mandate something resembling model cards for all AI systems sold or deployed in the EU. While the regulation does not specifically require the Mitchell et al. format, the overlap between regulatory requirements and model card best practices has led many organizations to use model cards as a starting point for compliance.

NIST AI Risk Management Framework

The US NIST AI RMF (AI 100-1), published in January 2023, organizes responsible AI practice around four core functions: GOVERN, MAP, MEASURE, and MANAGE. Documentation runs through all four. The MAP function calls for documenting context, risks, and intended uses; MEASURE calls for documenting evaluation methods and results; GOVERN calls for organizational policies that make documentation systematic; MANAGE calls for documenting risk responses and monitoring. The framework explicitly treats documentation as enabling transparency, accountability, and human review, even though it does not prescribe a specific template [14].

NIST has also published the Generative AI Profile (NIST AI 600-1, July 2024) that adapts the RMF specifically to generative models, with documentation expectations tailored to risks like data leakage, hallucination, and content provenance.

United States executive orders

In October 2023 the Biden administration issued Executive Order 14110, "Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence," which required developers of dual-use foundation models above specified compute thresholds to share safety test results with the federal government and contemplated reporting requirements that mirrored model card content. The order also created the US AI Safety Institute (USAISI) within NIST and tasked it with developing red-teaming guidelines [15].

On January 20, 2025, President Trump rescinded EO 14110 via Executive Order 14148. Three days later, Trump signed "Removing Barriers to American Leadership in Artificial Intelligence," which directs federal agencies to draft an AI Action Plan within 180 days. As of 2026, federal documentation requirements for foundation models in the United States are in flux, though state-level requirements (notably in Colorado) continue to apply [15][16].

Colorado AI Act

Colorado's Senate Bill 24-205, signed May 17, 2024 and known as the Colorado Anti-Discrimination in AI Law (ADAI), is the first state-level high-risk AI statute in the United States. It distinguishes between developers (who build or substantially modify high-risk AI systems) and deployers (who put them into use). Developers must give deployers a statement describing reasonably foreseeable uses, known harmful or inappropriate uses, the type of data used to train the system, and known or foreseeable limitations. They must also disclose any known risk of algorithmic discrimination to the Colorado Attorney General and to all known deployers within 90 days of discovery. Deployers must run impact assessments, maintain organized records (impact assessments, testing results, data source descriptions, version histories, internal review approvals), and notify consumers when a high-risk system is used to make a consequential decision about them. The provisions take effect June 30, 2026 [17].

Regulatory landscape at a glance

FrameworkJurisdictionIn forceDocumentation core
EU AI ActEuropean UnionAug 1, 2024 (GPAI obligations Aug 2, 2025)Technical documentation, training data summary, downstream provider info
NIST AI RMF 1.0United States (voluntary)Jan 26, 2023GOVERN/MAP/MEASURE/MANAGE functions, transparency woven through
NIST AI 600-1 (Gen AI Profile)United States (voluntary)Jul 26, 2024Generative-AI-specific risks and documentation
EO 14110 (Biden)United StatesOct 30, 2023 (rescinded Jan 20, 2025)Safety test reporting for dual-use models above compute thresholds
EO 14179 (Trump)United StatesJan 23, 2025Action plan; rolls back prior reporting requirements
Colorado SB 24-205 (ADAI)Colorado, USEffective Jun 30, 2026Developer/deployer documentation, impact assessments, consumer notice
Singapore Model AI Governance FrameworkSingapore (voluntary)2019 (rev. 2020, 2024)Documentation of design, data, and decisions
Canada Algorithmic Impact AssessmentCanada (federal use)2020Mandatory questionnaire for federal AI systems

What do real model cards look like?

The following table lists notable model cards and system cards published by major AI organizations, illustrating the range of documentation practices in use. Tables in this article are exempt from the one-link-per-article rule, so the same wikilinks may appear across rows.

OrganizationModelDocument typeYearNotable features
GoogleSmile detector, Toxicity (Perspective)Model card2019Original examples from the Mitchell et al. paper; revealed disparate performance for older men and for sexual-orientation terms
GoogleGemmaModel card2024Open-weights companion to Gemini; standardized template across Gemma 1 and 2
Google DeepMindGemini 1.0, 1.5, 2.0Technical report2023, 2024De facto system card; long-context evaluations and red-team results
OpenAIGPT-4System cardMarch 2023Red-team results across CBRN, persuasion, cybersecurity; bar exam at 90th percentile
OpenAIGPT-4V (vision)System cardSeptember 2023Image-input safety, person identification, biometric inference
OpenAIGPT-4oSystem cardAugust 2024Multimodal (text/image/audio); voice safety, speaker ID, ungrounded inference
OpenAIo1System cardDecember 2024First detailed evaluation of chain-of-thought reasoning safety
OpenAIOperatorSystem card2025Agentic browser use; documentation of containment and oversight
AnthropicClaude 3 (Opus, Sonnet, Haiku)Model cardMarch 2024Constitutional AI training, 14 policy areas in 6 languages
AnthropicClaude 3.5 Sonnet (and addendum)Model cardJune 2024, Oct 2024Document analysis, visual understanding, coding evaluations
AnthropicClaude 3.7 SonnetSystem card2025First Anthropic system card under updated RSP
AnthropicClaude Opus 4 / Sonnet 4System cardMay 2025Hybrid-reasoning models; CBRN and autonomy evaluations
AnthropicClaude Opus 4.5System cardNovember 2025RSP v3-aligned; Frontier Safety Roadmap and Risk Reports
MetaLlama 2Model cardJuly 2023Detailed benchmarks and Responsible Use Guide; published on Hugging Face
MetaLlama 3 (8B, 70B)Model cardApril 2024128K-token tokenizer, 8,192-token sequences; CyberSecEval and CBRNE testing
MetaLlama 3.1 / 3.2 / 3.3Model card2024Multilingual coverage, 405B variant, multimodal extensions
Mistral AIMistral 7B, Mixtral 8x7BModel card2023Concise template focused on benchmarks and Apache 2.0 license
Stability AIStable DiffusionModel card2022Documented training data sources, biases in image generation, misuse risks
DeepSeekDeepSeek-V3Technical reportDecember 2024671B MoE with 37B active; 14.8T training tokens; 2.788M H800 GPU hours
DeepSeekDeepSeek-R1Technical reportJanuary 2025Reinforcement-learning-only reasoning model with cold-start data

The variation between these documents reflects both organizational priorities and the absence of a binding standard. OpenAI and Anthropic have converged on long, multi-section system cards (often 50 to 200+ pages for frontier models, with the combined Claude Opus 4.6 and GPT-5.3 system cards reportedly totaling 244 pages). Meta and Mistral favor compact model cards. Open-source community releases sit anywhere on the spectrum.

What sections appear in modern LLM model cards?

For large language models, the standard model card template has expanded well beyond the original Mitchell et al. nine sections. A typical 2024 to 2026 LLM model card includes:

SectionTypical content
ArchitectureModel family, decoder-only or MoE, parameter count, active parameters, attention type (e.g., grouped-query, multi-head latent), context length
Training dataComposition (web, code, books, math, multilingual), token count, cutoff date, deduplication and filtering
Training computeFLOPs, hardware (e.g., H100, H800, TPU v5p), GPU-hours, training duration
Energy and carbonkWh consumed, location-based and market-based CO2e, sometimes water usage
Capabilities benchmarksMMLU, GPQA, HumanEval, MATH, BIG-Bench Hard, MMMU, AIME, SWE-bench, GDPval
Long-context evaluationsNeedle-in-a-haystack, RULER, multi-document QA
Safety evaluationsRed-team results, refusal rates, jailbreak resistance, sycophancy and deception checks
Bias and fairnessStereoSet, BBQ, CrowS-Pairs, multilingual fairness checks
Alignment evaluationsConstitutional AI compliance, instruction-following, RLHF win rates
Dangerous-capability evalsCBRN uplift, cybersecurity (CTF, vulnerability discovery), autonomous replication
Suggested usesRecommended applications and audiences
Out-of-scope usesApplications the developer disclaims (e.g., medical diagnosis, legal advice)
License and accessApache 2.0, MIT, Llama Community License, custom RAIL license, API-only access

Not every model card hits every section. Open-weights releases are more likely to publish carbon and compute numbers; closed-weights frontier releases tend to invest more in safety and dangerous-capability sections.

What are the limitations and criticisms of model cards?

Despite their wide adoption, model cards have been subject to several substantive critiques. Many of these limitations are acknowledged in the original Mitchell et al. paper, but they have not been fully resolved by subsequent practice.

Voluntary and inconsistent. Outside of regulated environments, model card creation is voluntary. Many models, particularly those released by smaller organizations or individual researchers, ship with incomplete or nonexistent documentation. The Liang et al. (2024) systematic analysis of 32,111 Hub model cards found that even the average card had multiple sections missing or stub-quality, with environmental, limitations, and evaluation sections least likely to be filled in [5].

Static documents. Model cards are typically created at the time of model release and rarely updated thereafter. As models are fine-tuned, deployed in new contexts, or discovered to have previously unknown failure modes, the original model card becomes outdated. Some organizations have adopted versioned model cards (Anthropic publishes addenda for each Claude release; Meta updates Llama cards across 3.1, 3.2, 3.3), but this practice is not widespread. Reward Reports (Gilbert et al., 2023) were proposed specifically to address the dynamic-system blind spot, focusing on reinforcement learning systems whose objectives evolve with deployment [18].

Self-reported. Model cards are written by the organizations that develop and release the models. There is an inherent tension between transparency and self-interest: organizations may understate risks, overstate performance, or omit information about known problems. Independent auditing of model card claims remains rare. Inioluwa Deborah Raji and colleagues have argued that internal algorithmic auditing should be standard practice and that external audit access should be a precondition for high-risk deployments [19].

Documentation theater. Critics have warned that producing a model card can substitute for actually doing the underlying safety, fairness, or accountability work. A polished card with the right section headers can give the impression of due diligence while the model itself was never tested in the relevant ways. This concern parallels longer-standing critiques of "ethics washing" in tech.

Readability and audience. Model cards are often written in technical language that is accessible to ML practitioners but opaque to policymakers, journalists, affected communities, and other stakeholders who may need the information most. The original Mitchell et al. paper acknowledged this tension and called for plain-language summaries; in practice, most cards still read like research-paper appendices.

Scope limitations. A model card documents a model; it does not document the deployment context, the downstream applications, or the lived experiences of people affected by the model's outputs. A toxicity classifier's model card might document that the model performs well on a standard benchmark, without capturing how its deployment in a content moderation system affects free expression for specific communities. System cards partially address this gap, but deployment-level documentation remains underdeveloped.

Benchmark gaming. As model cards have become competitive marketing artifacts, the benchmarks they report have become subject to contamination, cherry-picking, and tuning-for-the-test. The cards themselves are not at fault, but they have helped concentrate attention on a narrow set of metrics that may not generalize.

How does the broader ML documentation landscape fit together?

Model cards exist within a broader ecosystem of documentation and governance tools for machine learning. Hugging Face maintains a Landscape of ML Documentation Tools resource that catalogs these complementary frameworks; each addresses a different layer of the AI lifecycle.

Tool / FrameworkPurposeOriginating work
Model cardsDocument individual ML modelsMitchell et al., FAT* 2019
Datasheets for datasetsDocument training and evaluation datasetsGebru et al., CACM 2021
System cardsDocument deployed AI systems including safety measuresOpenAI (GPT-4 System Card, 2023)
AI FactSheetsComprehensive AI governance documentationArnold et al., IBM 2019
Data StatementsDocument characteristics of NLP datasetsBender and Friedman, TACL 2018 (v3 2023)
Reward ReportsDocument objectives and reward functions of reinforcement learning systemsGilbert et al., AIES 2023
Dataset Nutrition LabelsStandardized nutrition-label-style data summaryHolland et al., 2018
Responsible AI Licenses (RAIL)License terms with use restrictions for AI modelsRAIL Initiative
Algorithmic Impact AssessmentsPre-deployment risk assessment of AI systemsGovernment of Canada, 2020

These tools serve overlapping but distinct purposes. No single document can capture all relevant information about an AI system, from raw training data to real-world impact. The trend in AI documentation is toward complementary, layered documentation that covers different aspects of the AI lifecycle, with model cards anchoring the model layer while datasheets cover data, system cards cover deployment, and reward reports cover dynamic behavior.

What is the current state and outlook for model cards?

As of 2026, model cards are firmly established as a norm in the AI industry, though their adoption and quality remain uneven. The EU AI Act's GPAI obligations have been in force since August 2025, and full enforcement powers begin August 2026, which is already pulling the median quality of frontier-model documentation upward. Providers selling into the European market increasingly publish documentation that satisfies EU technical-documentation expectations regardless of where the company itself sits.

Hugging Face continues to lead in model card tooling and infrastructure. The platform now hosts more than two million public models, with automated card generation features that pre-populate certain fields based on metadata, training logs, and benchmark results [24]. Research groups have explored using language models themselves to draft model card content, an idea that raises awkward questions about accuracy and accountability when the documenter is the same kind of system being documented.

Frontier-lab system cards have grown dramatically in scope. Where the GPT-4 System Card in 2023 ran roughly 60 pages, more recent releases such as Claude Opus 4.5 and the GPT-5.3 system cards run well over 100 pages each, with detailed sections on dangerous-capability evaluations (CBRN uplift studies, cyber CTF performance, autonomous replication tests), agentic-task evaluations like SWE-bench and OpenAI's GDPval gold dataset of 220 tasks across 44 occupations, and a growing focus on evaluation reproducibility. Anthropic's Responsible Scaling Policy, OpenAI's Preparedness Framework, and Google DeepMind's Frontier Safety Framework all tie release decisions to specific evaluations that get reported back in the system card.

The broader trajectory points toward AI documentation becoming not just a best practice but a legal and commercial requirement. As AI systems are deployed in healthcare, finance, criminal justice, and defense, the demand for thorough, accurate, and independently verifiable documentation will only grow. Model cards, for all their limitations, established the vocabulary and expectations that more rigorous successors are now being built on.

See also

References

  1. Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I.D., and Gebru, T. "Model Cards for Model Reporting." Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* 2019), 220-229. https://arxiv.org/abs/1810.03993 and https://dl.acm.org/doi/10.1145/3287560.3287596
  2. Mitchell, M. Personal site and biography. https://www.m-mitchell.com/ ; "Margaret Mitchell (scientist)." Wikipedia. https://en.wikipedia.org/wiki/Margaret_Mitchell_(scientist)
  3. Hugging Face. "Model Cards." https://huggingface.co/docs/hub/en/model-cards ; Hugging Face. "Model Card Guidebook." https://huggingface.co/docs/hub/en/model-card-guidebook ; Hugging Face. "Annotated Model Card Template." https://huggingface.co/docs/hub/en/model-card-annotated
  4. Hugging Face. "huggingface_hub v1.0: Five Years of Building the Foundation of Open Machine Learning." Blog post, 2025. https://huggingface.co/blog/huggingface-hub-v1
  5. Liang, W., Rajani, N., Yang, X., Ozoani, E., Wu, E., Chen, Y., Smith, D.S., and Zou, J. "Systematic analysis of 32,111 AI model cards characterizes documentation practice in AI." Nature Machine Intelligence, 2024. https://www.nature.com/articles/s42256-024-00857-z ; preprint "What's documented in AI? Systematic Analysis of 32K AI Model Cards." https://arxiv.org/abs/2402.05160
  6. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wallach, H., Daume III, H., and Crawford, K. "Datasheets for Datasets." Communications of the ACM, Vol. 64, No. 12, December 2021, 86-92. https://arxiv.org/abs/1803.09010 and https://dl.acm.org/doi/10.1145/3458723
  7. Lunden, I. "Google fires top AI ethics researcher Margaret Mitchell." TechCrunch, February 19, 2021. https://techcrunch.com/2021/02/19/google-fires-top-ai-ethics-researcher-margaret-mitchell/
  8. OpenAI. "GPT-4 System Card." March 23, 2023. https://cdn.openai.com/papers/gpt-4-system-card.pdf ; OpenAI. "GPT-4o System Card." August 2024. https://openai.com/index/gpt-4o-system-card/ ; OpenAI. "Operator System Card." 2025. https://openai.com/index/operator-system-card/
  9. Anthropic. "The Claude 3 Model Family: Opus, Sonnet, Haiku." Model card, March 2024. https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf ; Anthropic. "Model system cards." https://www.anthropic.com/system-cards
  10. Anthropic. "Anthropic's Responsible Scaling Policy." Versions 1.0 (Sept 2023), 2.0 (Oct 2024), 3.0 (2025). https://www.anthropic.com/responsible-scaling-policy
  11. DeepSeek-AI. "DeepSeek-V3 Technical Report." arXiv:2412.19437, December 2024. https://arxiv.org/abs/2412.19437 ; DeepSeek-AI. "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning." January 2025. https://huggingface.co/deepseek-ai/DeepSeek-R1
  12. European Commission. "AI Act." Regulation (EU) 2024/1689. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai ; European Commission. "AI Act enters into force." August 1, 2024. https://commission.europa.eu/news-and-media/news/ai-act-enters-force-2024-08-01_en ; European Commission. "The General-Purpose AI Code of Practice." https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai
  13. European Commission. "EU rules on general-purpose AI models start to apply." August 2025. https://digital-strategy.ec.europa.eu/en/news/eu-rules-general-purpose-ai-models-start-apply-bringing-more-transparency-safety-and-accountability ; Future of Life Institute. "High-level summary of the AI Act." https://artificialintelligenceact.eu/high-level-summary/
  14. National Institute of Standards and Technology. "Artificial Intelligence Risk Management Framework (AI RMF 1.0)." NIST AI 100-1, January 2023. https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf ; NIST. "AI Risk Management Framework: Generative AI Profile." NIST AI 600-1, July 2024. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
  15. White House. "Executive Order 14110: Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence." October 30, 2023. https://en.wikipedia.org/wiki/Executive_Order_14110
  16. White House. "Executive Order 14148: Initial Rescissions of Harmful Executive Orders and Actions." January 20, 2025; "Removing Barriers to American Leadership in Artificial Intelligence." January 23, 2025. https://www.aila.org/library/executive-order-on-removing-barriers-to-american-leadership-in-artificial-intelligence
  17. Colorado General Assembly. "SB24-205 Consumer Protections for Artificial Intelligence." Signed May 17, 2024, effective June 30, 2026. https://leg.colorado.gov/bills/sb24-205
  18. Gilbert, T.K., Lambert, N., Dean, S., Zick, T., Snoswell, A., and Mehta, S. "Reward Reports for Reinforcement Learning." Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. https://arxiv.org/abs/2204.10817
  19. Raji, I.D., Smart, A., White, R.N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., and Barnes, P. "Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing." FAT* 2020. https://dl.acm.org/doi/10.1145/3351095.3372873
  20. Bender, E.M. and Friedman, B. "Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science." Transactions of the Association for Computational Linguistics, 2018. https://aclanthology.org/Q18-1041/
  21. Arnold, M., Bellamy, R.K.E., Hind, M., et al. "FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity." IBM Journal of Research and Development. https://research.ibm.com/blog/aifactsheets
  22. TensorFlow / Google. "Model Card Toolkit." https://github.com/tensorflow/model-card-toolkit ; "Introducing the Model Card Toolkit for Easier Model Transparency Reporting." Google Research blog, July 2020. https://research.google/blog/introducing-the-model-card-toolkit-for-easier-model-transparency-reporting/
  23. Meta. "Llama 3 Model Card." April 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md ; "Llama 3.1 Model Card." https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md
  24. Hugging Face. "huggingface_hub v1.0: Five Years of Building the Foundation of Open Machine Learning." 2025. https://huggingface.co/blog/huggingface-hub-v1 ; Castano, J. et al. "Anatomy of a Machine Learning Ecosystem: 2 Million Models on Hugging Face." arXiv:2508.06811, 2025. https://arxiv.org/abs/2508.06811

Improve this article

Add missing citations, update stale details, or suggest a clearer explanation. Every suggestion is reviewed for sourcing before it goes live.

4 revisions by 1 contributors · full history

Suggest edit