Common Corpus
Common Corpus is the largest fully open, multilingual dataset for pretraining large language models, assembled and released by the French AI research lab Pleias.
Open Source AI
Explore Open Source AI through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Data & Datasets
Articles that also belong to these categories. Counts cover all of Open Source AI.
All articles (444)Large Language Models (165)AI Models (120)Developer Tools (109)Chinese AI (77)AI Agents (53)AI Companies (42)Mixture of Experts (40)AI Infrastructure (37)Multimodal AI (28)NVIDIA (28)Robotics (28)Machine Learning (25)AI Code Generation (23)Natural Language Processing (23)AI Tools & Products (18)Generative AI (18)Meta AI (17)Small Language Models (16)Data & Datasets (15)Video Generation (13)Computer Vision (12)Diffusion Models (12)Image Generation (12)Information Retrieval (12)Google (11)Reasoning Models (11)AI Research (10)Artificial Intelligence (10)AI Hardware (9)AI Inference (9)Microsoft (9)Research Organizations (9)Speech & Audio AI (9)Embodied AI (8)Google DeepMind (8)MLOps (7)Training & Optimization (7)AI for Science (6)Reinforcement Learning (6)Robot Hardware (6)Deep Learning (5)Drug Discovery (5)OpenAI (5)Voice AI (5)AI Benchmarks (4)AI Safety (4)Healthcare AI (4)Model Evaluation (4)Algorithms (3)Conversational AI (3)AI Ethics (2)Agentic Commerce (2)Anthropic (2)Enterprise AI (2)Humanoid Robots (2)Interpretability (2)Mathematics (2)Model Architecture (2)Neural Networks (2)People (2)Physical AI (2)Programming Languages (2)Software Development (2)Transformer Models (2)AI Incidents & Controversies (1)AI Policy (1)AI Policy & Regulation (1)Data Science (1)Music & Audio Generation (1)Quantum Computing (1)Robotics Companies (1)World Models (1)
Showing 1-15 of 15 articles
Common Pile
Common Pile v0.1 is an 8 terabyte corpus of openly licensed and public domain text, released on June 5, 2025, by EleutherAI and a consortium of more than two dozen academic and industry collaborators.
Cosmopedia
Cosmopedia is an open synthetic pretraining dataset released by Hugging Face in February 2024, made up of textbooks, blog posts, stories, and WikiHow-style articles written entirely by a large language model.
DCLM (DataComp for Language Models)
DCLM, short for DataComp for Language Models (also styled DataComp-LM), is an open benchmark, dataset, and software framework, released in June 2024
Dolma
Dolma is an open three-trillion-token English pretraining corpus released by the Allen Institute for AI (AI2) to power its fully open OLMo language models and to let researchers study how training data shapes…
FineWeb
FineWeb is a large-scale, open pretraining dataset for large language models (LLMs) created by Hugging Face.
NuminaMath
NuminaMath is a family of openly licensed competition-mathematics resources developed by the non-profit Project Numina, spanning the largest public dataset of competition math problems and solutions, a set of…
OpenOrca
OpenOrca is a large open-source instruction-tuning dataset that augments the FLAN Collection with chain-of-thought responses generated by OpenAI's GPT-3.5 and GPT-4 APIs.
RedPajama
RedPajama is a family of large-scale, openly licensed datasets for training large language models (LLMs), created by Together AI with academic and open-source partners to reproduce, in fully open form
RefinedWeb
RefinedWeb is a large-scale English pretraining dataset for large language models, built from filtered and deduplicated Common Crawl web data alone and released in June 2023 by the Technology Innovation…
SlimPajama
SlimPajama is a 627-billion-token English-language pre-training corpus for large language models, produced by Cerebras Systems in collaboration with the Opentensor Foundation by extensively cleaning and…
Snorkel
Snorkel is an open-source software framework and methodology for programmatic data labeling that started at Stanford University in 2015 and is licensed under Apache 2.0 .
The Pile (dataset)
The Pile is an 825.18 GiB (approximately 886 GB) English text corpus designed for training large language models, assembled from 22 diverse, high-quality subsets spanning academic, professional, internet…
The Stack (BigCode dataset)
The Stack is a family of large, permissively-licensed source-code datasets built by the BigCode project, an open scientific collaboration jointly led by Hugging Face and ServiceNow Research, to train and…
TxT360
TxT360 is an open large-scale pretraining corpus for large language models, released in October 2024 by the LLM360 project, a collaboration led by Petuum and the Mohamed bin Zayed University of Artificial…