CtrlK
Model collapse is a degenerative phenomenon in which generative AI systems progressively lose the ability to represent the full diversity of their original training distribution when they are trained on data produced by other AI models. First formally characterized in a 2024 Nature paper by Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal, model collapse occurs because each generation of model-generated data introduces small approximation errors that compound over successive training cycles [1]. Over time, rare events and low-probability features in the data distribution are systematically lost, and the model's outputs converge toward a narrow, homogeneous subset of what the original distribution contained. The phenomenon has been demonstrated in large language models, variational autoencoders (VAEs), and Gaussian mixture models (GMMs), and it has raised significant concerns about the long-term viability of training AI systems on internet-scale data as the web becomes increasingly saturated with AI-generated content.
The rapid proliferation of generative AI tools since 2022, including ChatGPT, Stable Diffusion, and other systems built on large language models and diffusion models, has led to an explosion of AI-generated text, images, audio, and video on the internet. Estimates from various research groups suggest that a substantial and growing fraction of publicly available web content is now machine-generated. This creates a feedback loop: new AI models are typically trained on data scraped from the web, which increasingly contains outputs from previous AI models. The central question motivating model collapse research is what happens when this cycle repeats across multiple generations of models.
Before the Shumailov et al. paper, several research groups had observed symptoms of what would later be called model collapse. The precursor paper, "The Curse of Recursion: Training on Generated Data Makes Models Forget," posted as a preprint in May 2023 by many of the same authors, introduced the core concept and provided early experimental evidence [2]. The 2024 Nature publication formalized the theoretical framework, expanded the experimental scope, and brought the phenomenon to wide attention in both the research community and the general public.
Model collapse unfolds through a process that can be understood in several stages. At its core, the problem is one of compounding approximation errors across generations of model training.
Consider a sequence of generative models, where model 0 is trained on authentic human-generated data, model 1 is trained on data generated by model 0, model 2 is trained on data generated by model 1, and so on. Each model in this chain learns an approximation of the data distribution it was trained on. No model perfectly captures every feature of its training data; there are always small errors introduced by finite sample sizes, limited model capacity, and the stochastic nature of training.
When the next model in the chain trains on the outputs of the previous model, it learns from data that already contains these approximation errors. The new model then introduces its own approximation errors on top of those inherited from its predecessor. With each successive generation, the errors accumulate and amplify.
The first visible effect is what Shumailov et al. term "early model collapse." In this stage, the model begins to lose information about the tails of the data distribution. The tails represent rare events, unusual patterns, minority viewpoints, and low-probability outcomes. These are precisely the features that are most likely to be underrepresented in any finite sample drawn from the distribution.
When model 0 generates a dataset, rare events that occur with probability less than approximately 1/M (where M is the number of samples generated) are unlikely to appear in that dataset at all. Model 1, trained on this dataset, has no opportunity to learn these rare patterns. Even events that do appear in the generated dataset may be underrepresented relative to their true frequency, causing the next model to assign them even lower probability. Over successive generations, this trimming effect propagates inward from the tails, progressively eliminating features that were increasingly common in the original distribution.
If the recursive training process continues long enough, the models enter "late model collapse." In this stage, the learned distribution has lost so much information that it bears little resemblance to the original data distribution. The variance of the distribution shrinks dramatically, and the model's outputs converge toward a narrow cluster of high-probability outputs. In the most extreme cases, the model effectively learns to produce a single mode or a small set of nearly identical outputs, regardless of the prompt or conditioning.
The mathematical explanation of model collapse rests on the statistical properties of sampling and density estimation across multiple generations.
Suppose the true data distribution P has a probability density function p(x). When we draw M samples from P to create a training set, the empirical distribution P_hat approximates P but necessarily loses information about regions where p(x) is very small. Specifically, any event with probability less than about 1/M is likely to be completely absent from the sample. This is not a failure of the sampling procedure; it is a fundamental statistical limitation.
When a generative model is trained on these M samples, it learns an approximation Q of P_hat, which is itself an approximation of P. The model Q may further smooth, distort, or truncate the distribution due to its own inductive biases and capacity limitations.
Let P_0 = P be the original distribution, and let P_n be the distribution learned by the nth generation model. Each generation introduces an error term. The total error after n generations can be decomposed into two components [1]:
Shumailov et al. showed that for distributions with different tail characteristics (light-tailed vs. heavy-tailed), the resampling process causes the observed distributions to converge. Although the original distributions may be very different, after sufficient rounds of resampling and refitting, they become indistinguishable because the tails that differentiated them have been eliminated [1].
The mathematical analysis reveals that tail behavior is the key factor determining vulnerability to model collapse. Distributions with heavier tails (such as power-law or Pareto distributions) lose more information per generation because their tails extend further into low-probability regions. Light-tailed distributions (such as Gaussian distributions) are somewhat more resilient but still degrade over sufficient generations.
The critical insight is that the information lost in each generation is not random; it is systematically biased toward rare events. This means that model collapse disproportionately affects the representation of minority groups, unusual patterns, creative outliers, and rare but important knowledge.
| Distribution Type | Tail Behavior | Vulnerability to Model Collapse | Information Loss Pattern |
|---|---|---|---|
| Gaussian (light-tailed) | Exponential decay | Moderate | Gradual variance reduction |
| Power-law (heavy-tailed) | Polynomial decay | High | Rapid tail trimming |
| Bounded distributions | Hard cutoff | Lower (but still present) | Edge erosion |
| Mixture models | Multiple modes | High | Minor modes disappear first |
Shumailov et al. provided experimental evidence of model collapse across three different types of generative models, demonstrating that the phenomenon is not specific to a single architecture but is a general property of recursive training on synthetic data.
The simplest experimental setting involved Gaussian mixture models (GMMs). A GMM was first fitted to a dataset, then used to generate synthetic data, which was used to fit a new GMM, and so on. After several generations, the fitted GMMs showed dramatically reduced variance, with modes collapsing toward the overall mean. Minor modes (representing less frequent clusters in the original data) disappeared first, followed by progressive convergence of the remaining modes. This experiment served as a clean illustration of the mathematical principles because GMMs have well-understood statistical properties.
Experiments with variational autoencoders (VAEs) trained on image data showed similar degradation. When VAEs were recursively trained on their own generated images, the quality and diversity of generated images decreased with each generation. Fine details were lost first, followed by broader structural features. After several generations, the generated images became blurry and homogeneous, lacking the variety present in the original training set.
The most consequential experiments involved large language models. The researchers fine-tuned OPT-125M, an open-source language model released by Meta, in a recursive chain where each generation was trained on text produced by its predecessor. The training data was based on the WikiText-2 dataset [1].
The results were striking. In one experiment, after an initial English-language input about medieval architecture, later generations of the model produced outputs about completely unrelated topics, such as jackrabbits with different-colored tails, demonstrating severe topic drift. More systematically, the researchers measured perplexity (a standard metric of language model quality, where lower values indicate better performance) and found that it increased steadily across generations, indicating progressive degradation in the model's ability to produce coherent, high-quality text.
| Generation | Observed Behavior | Perplexity Trend |
|---|---|---|
| 0 (original) | Coherent, diverse text matching training distribution | Baseline |
| 1-2 | Slight loss of rare vocabulary and unusual phrasings | Modest increase |
| 3-5 | Noticeable reduction in topic diversity; repetitive patterns emerge | Significant increase |
| 5-9 | Severe topic drift; nonsensical or irrelevant outputs | Steep increase |
| 9+ | Near-total collapse; outputs unrelated to original distribution | Very high |
The Shumailov et al. findings prompted significant follow-up research, including both supporting studies and critical responses.
A 2025 paper published at ICLR, "Strong Model Collapse," extended the theoretical analysis and provided additional experimental evidence [3]. The paper demonstrated that model collapse can occur even when models have access to a mixture of real and synthetic data, although the rate of collapse depends on the proportion of synthetic data in the training mix.
Researchers at various institutions replicated the core findings using different model architectures and datasets, confirming that model collapse is a robust phenomenon rather than an artifact of specific experimental choices.
Ali Borji published a detailed critique in October 2024, "A Note on Shumailov et al. (2024)," arguing that some aspects of the experimental setup may not reflect real-world training practices [4]. The critique noted that actual AI training pipelines typically involve data curation, filtering, and mixing of multiple data sources rather than naive recursive training on a single model's outputs. Other researchers pointed out that the severity of model collapse depends heavily on the ratio of synthetic to real data in the training mix, and that careful data curation can substantially mitigate the effect.
A key finding from the paper "Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data" (2024) showed that if real data is accumulated alongside synthetic data rather than replaced by it, model collapse can be avoided [5]. This result suggests that model collapse is not an inevitable fate but rather a consequence of specific (and avoidable) data management practices.
Model collapse has several important implications for the AI industry and research community.
The most immediate concern is that the internet, which has been the primary source of training data for large AI models, is becoming contaminated with AI-generated content at an accelerating pace. Common Crawl, the web-scraping dataset used as a foundation for many large language models, does not currently have reliable mechanisms for distinguishing human-generated content from AI-generated content. As the proportion of synthetic content on the web grows, future models trained on web-scraped data will increasingly be training on outputs of previous models, creating exactly the conditions that lead to model collapse.
Model collapse has elevated the value of datasets known to contain only human-generated content, particularly those collected before the widespread deployment of generative AI tools (roughly pre-2022). Organizations and researchers have begun treating these "pre-AI" datasets as particularly valuable resources. Some have advocated for the creation of curated data repositories with strong provenance guarantees [6].
Because model collapse preferentially eliminates rare events and low-probability patterns, it poses particular risks for the representation of minority groups, less common languages, specialized domains, and creative outliers. If models progressively lose the ability to generate or understand content from the tails of the distribution, the resulting AI systems may become less useful for users whose needs or perspectives are not part of the dominant mode.
As generative AI systems play an increasingly central role in knowledge retrieval and content creation, model collapse raises the specter of gradual knowledge loss. Obscure but accurate information, rare cultural references, specialized technical knowledge, and other forms of "long-tail" content could be progressively lost as models collapse toward the most common patterns in their training data.
Researchers and practitioners have proposed several strategies for preventing or mitigating model collapse.
The most straightforward mitigation is to ensure that training sets always include a substantial fraction of verified human-generated data. Shumailov et al. showed that access to even a modest amount of original human data can significantly slow or prevent model collapse [1]. Several organizations have begun investing in high-quality human annotation and data collection programs specifically to maintain this anchor.
Knowing whether a piece of training data was generated by a human or a machine is essential for preventing model collapse. Data provenance systems track the origin, history, and transformations of training data. Technologies such as AI watermarking, content credentials (C2PA), and metadata tagging can help identify synthetic content in training pipelines [6]. Strong provenance practices include recording source URLs, collection dates, generation flags (human-origin, machine-origin, or unknown), and licensing information.
Rather than excluding all synthetic data, some researchers advocate for careful curation and quality filtering. The key insight is that not all synthetic data is harmful; the problem arises from indiscriminate mixing of synthetic and real data without quality controls. Research has shown that training on filtered synthetic data can not only avoid model degradation but can sometimes enhance performance relative to training on unfiltered data [7]. Filtering strategies include deduplication, quality scoring, diversity checks, and comparison against known real-data benchmarks.
Research on the optimal mixing of real and synthetic data has shown that there exist ratios at which the benefits of additional data (even synthetic data) outweigh the risks of model collapse. A 2024 study derived theoretical bounds on the optimal fraction of synthetic data as a function of the total dataset size and the quality of the synthetic data [5]. The practical recommendation is to keep synthetic data as a supplement to, not a replacement for, real data, and to control its proportion carefully.
A 2025 approach proposed "escaping model collapse via synthetic data verification," where synthetic data is validated against quality criteria before being included in training sets [8]. This can involve checking the synthetic data for statistical consistency with known real-data distributions, evaluating its diversity, and removing samples that show signs of mode collapse or other degradation.
Continuous monitoring of model performance on distribution tails is an important practical mitigation. By evaluating models on slice-based benchmarks that specifically test performance on rare events, minority categories, and low-frequency patterns, practitioners can detect early signs of model collapse before it becomes severe. This approach allows for corrective action (such as augmenting training data with more real examples from underrepresented categories) before the damage becomes irreversible.
| Mitigation Strategy | Mechanism | Effectiveness | Practical Challenges |
|---|---|---|---|
| Real data anchoring | Maintain fraction of verified human data | High (prevents collapse when sufficient) | Sourcing and verifying human data at scale |
| Data provenance tracking | Identify and label synthetic content | Moderate to high | No universal standard; watermarks can be stripped |
| Synthetic data filtering | Remove low-quality synthetic samples | Moderate | Requires reliable quality metrics |
| Optimal mixing ratios | Control synthetic/real data proportion | High (when ratios are well-calibrated) | Optimal ratios depend on domain and model |
| Tail performance monitoring | Detect early collapse on rare categories | Moderate (detection, not prevention) | Requires representative evaluation sets |
| Data verification | Validate synthetic data before inclusion | Moderate to high | Computationally expensive at scale |
Model collapse is related to several other phenomena in machine learning and statistics.
Mode collapse in generative adversarial networks (GANs) is a well-known training failure where the generator produces only a small subset of the possible outputs, ignoring other modes of the data distribution. While the mechanism is different (mode collapse in GANs arises from the adversarial training dynamics rather than recursive training on synthetic data), the outcome is similar: a loss of diversity in generated outputs. Model collapse can be seen as a multi-generational analog of mode collapse.
Catastrophic forgetting occurs when a neural network trained on a new task loses its ability to perform previously learned tasks. Model collapse involves a related but distinct form of forgetting: the model loses its knowledge of the tails of the distribution rather than entire tasks. Both phenomena reflect the limited capacity of neural networks to retain information when exposed to new data that does not adequately represent what came before.
In traditional machine learning operations (MLOps), data drift refers to changes in the distribution of input data over time, which can degrade model performance. Model collapse can be viewed as a self-inflicted form of data drift, where the model's own outputs contaminate future training data, causing the training distribution to shift away from the original real-world distribution.
As of early 2026, model collapse is a widely recognized risk in the AI research and industry communities, but it is not yet a fully solved problem.
The consensus view is that naive recursive training on model-generated data without real-data anchoring will lead to collapse, and this has been confirmed across multiple model types and experimental settings. However, the practical severity of the problem depends heavily on how training data is managed. Real-world AI training pipelines already incorporate some degree of data curation, filtering, and mixing, which provides partial protection against model collapse.
Major AI companies have responded by investing in data provenance infrastructure, curating high-quality human-generated datasets, and developing watermarking and detection tools to identify AI-generated content in their training pipelines. The EU AI Act, which includes transparency requirements for AI-generated content labeling and watermarking that become enforceable in August 2026, is expected to create additional incentives for provenance tracking that could help mitigate model collapse risks [9].
However, several challenges remain. There is no universal standard for identifying AI-generated content, and existing detection tools have significant limitations. The proportion of AI-generated content on the internet continues to grow, and it is unclear whether current mitigation strategies will be sufficient at the scale of the entire web. Research published at ICLR 2025 confirmed that the risk of model collapse is not yet fully addressed, even as practical mitigations continue to improve [3].
The model collapse problem has also prompted broader reflection on the sustainability of current approaches to AI training. Some researchers have argued that it highlights the need for fundamentally new approaches to data collection and model training, rather than relying on ever-larger web scrapes. Others have pointed out that model collapse is ultimately a problem of information loss, and that solving it requires treating training data as a carefully managed resource rather than an abundant commodity.