Cosmopedia
Cosmopedia is an open synthetic pretraining dataset released by Hugging Face in February 2024, made up of textbooks, blog posts, stories, and WikiHow-style articles written entirely by a large language model.
Data & Datasets
Explore Data & Datasets through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Large Language Models
Articles that also belong to these categories. Counts cover all of Data & Datasets.
All articles (116)Machine Learning (65)Natural Language Processing (15)Open Source AI (15)Computer Vision (13)AI Benchmarks (11)Large Language Models (11)Statistics (10)AI Ethics (6)Meta AI (6)Robotics (5)Embodied AI (4)AI Companies (3)Model Evaluation (3)NVIDIA (3)AI Research (2)Data Science (2)Deep Learning (2)MLOps (2)Multimodal AI (2)Training & Optimization (2)AI Code Generation (1)AI History (1)AI Infrastructure (1)AI Models (1)AI Policy & Regulation (1)AI Safety (1)AI Tools & Products (1)AI for Science (1)Algorithms (1)Artificial Intelligence (1)Chinese AI (1)Developer Tools (1)Enterprise AI (1)Generative AI (1)Google (1)Google DeepMind (1)Mathematics (1)Robotics Companies (1)
Showing 1-11 of 11 articles
Dolma
Dolma is an open three-trillion-token English pretraining corpus released by the Allen Institute for AI (AI2) to power its fully open OLMo language models and to let researchers study how training data shapes…
FineWeb-Edu
FineWeb-Edu is an open, English-language pretraining dataset of roughly 1.3 trillion tokens
How to Prevent OpenAI and Google From Training Their LLMs on Your Website's Data
You can stop OpenAI, Google, Anthropic, and most other major AI companies from using your website to train large language models (LLMs) by adding a small set of directives to your site's robots.txt file.
OpenOrca
OpenOrca is a large open-source instruction-tuning dataset that augments the FLAN Collection with chain-of-thought responses generated by OpenAI's GPT-3.5 and GPT-4 APIs.
RefinedWeb
RefinedWeb is a large-scale English pretraining dataset for large language models, built from filtered and deduplicated Common Crawl web data alone and released in June 2023 by the Technology Innovation…
Self-Instruct
Self-Instruct is a semi-automated framework for aligning a pretrained large language model with instruction-following behavior by bootstrapping its training data from the model itself, introduced in December…
SlimPajama
SlimPajama is a 627-billion-token English-language pre-training corpus for large language models, produced by Cerebras Systems in collaboration with the Opentensor Foundation by extensively cleaning and…
TxT360
TxT360 is an open large-scale pretraining corpus for large language models, released in October 2024 by the LLM360 project, a collaboration led by Petuum and the Mohamed bin Zayed University of Artificial…
UltraChat
UltraChat is a large-scale synthetic multi-turn instructional conversation dataset released in May 2023 by the OpenBMB group at Tsinghua University, comprising approximately 1.5 million dialogues generated by…
WRAP (Web Rephrase Augmented Pre-training)
WRAP (Web Rephrase Augmented Pre-training) is a synthetic-data pre-training method introduced in the paper "Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling," posted to arXiv on 29…