Concept dictionary
Model Collapse
Model collapse is degradation that can occur when models are repeatedly trained on generated data in ways that cause the learned distribution to lose important variation, accuracy, or representation of rare cases.
- Review state
- Reviewed in POC
- Freshness
- Current
- Version
- 2
Also called
- Model Collapse
Remember it this wayCopies of copies can lose the original variation.
In plain language
If generated material replaces too much direct or well-curated evidence, later models may learn a narrower and increasingly distorted version of the original patterns.
Why it matters
Synthetic data can be useful, but its origin, quality, proportions, and effects need controls. Repeated reuse can hide what information has been lost.
Example
Successive image models are trained largely on outputs from earlier models and gradually reproduce a narrower range of features while rare patterns disappear.
Common misconception
Model collapse is not an inevitable result of any synthetic data. It describes risks from particular recursive and poorly controlled training conditions.
Related terms
Sources
- Shumailov et al., AI models collapse when trained on recursively generated dataprimary empirical paper on model collapse from recursive synthetic-data training.
Learn more
- Track generated inputs with [[data-provenance|Data Provenance]] and distinguish this from [[model-drift|Model Drift]].