Cosmopedia
Cosmopedia is an open synthetic pretraining dataset released by Hugging Face in February 2024, made up of textbooks, blog posts, stories, and WikiHow-style articles written entirely by a large language model.
Large Language Models
Explore language models, how they work, and the techniques used to build applications with them.
Explore articles
Browse subtopics: Data & Datasets
Articles that also belong to these categories. Counts cover all of Large Language Models.
All articles (545)AI Models (184)Open Source AI (165)Chinese AI (92)Natural Language Processing (58)AI Companies (53)Machine Learning (49)Mixture of Experts (46)Multimodal AI (41)Reasoning Models (39)Artificial Intelligence (37)OpenAI (35)AI Benchmarks (33)Anthropic (31)Google DeepMind (26)AI Agents (22)AI Safety (21)Developer Tools (20)Meta AI (20)AI Code Generation (18)Small Language Models (18)Training & Optimization (18)AI Inference (16)Model Architecture (14)Generative AI (13)NVIDIA (13)Prompt Engineering (13)AI Research (12)Deep Learning (12)Google (12)Model Evaluation (12)Data & Datasets (11)AI Tools & Products (9)Conversational AI (9)Microsoft (9)Transformer Models (9)AI Infrastructure (7)Diffusion Models (7)AI Alignment (5)Enterprise AI (5)Research Organizations (5)AI Hardware (4)Healthcare AI (4)Information Retrieval (4)Interpretability (4)People (4)Mathematics (3)AI History (2)AI for Science (2)Algorithms (2)Computer Vision (2)Legal AI (2)Reinforcement Learning (2)Speech & Audio AI (2)AI Policy (1)AI Policy & Regulation (1)AI in Cybersecurity (1)AI in Gaming (1)Autonomous Vehicles (1)ChatGPT (1)China AI (1)Drug Discovery (1)Neural Networks (1)Robotics (1)Software Development (1)Video Generation (1)
Showing 1-11 of 11 articles
Dolma
Dolma is an open three-trillion-token English pretraining corpus released by the Allen Institute for AI (AI2) to power its fully open OLMo language models and to let researchers study how training data shapes…
FineWeb-Edu
FineWeb-Edu is an open, English-language pretraining dataset of roughly 1.3 trillion tokens
How to Prevent OpenAI and Google From Training Their LLMs on Your Website's Data
You can stop OpenAI, Google, Anthropic, and most other major AI companies from using your website to train large language models (LLMs) by adding a small set of directives to your site's robots.txt file.
OpenOrca
OpenOrca is a large open-source instruction-tuning dataset that augments the FLAN Collection with chain-of-thought responses generated by OpenAI's GPT-3.5 and GPT-4 APIs.
RefinedWeb
RefinedWeb is a large-scale English pretraining dataset for large language models, built from filtered and deduplicated Common Crawl web data alone and released in June 2023 by the Technology Innovation…
Self-Instruct
Self-Instruct is a semi-automated framework for aligning a pretrained large language model with instruction-following behavior by bootstrapping its training data from the model itself, introduced in December…
SlimPajama
SlimPajama is a 627-billion-token English-language pre-training corpus for large language models, produced by Cerebras Systems in collaboration with the Opentensor Foundation by extensively cleaning and…
TxT360
TxT360 is an open large-scale pretraining corpus for large language models, released in October 2024 by the LLM360 project, a collaboration led by Petuum and the Mohamed bin Zayed University of Artificial…
UltraChat
UltraChat is a large-scale synthetic multi-turn instructional conversation dataset released in May 2023 by the OpenBMB group at Tsinghua University, comprising approximately 1.5 million dialogues generated by…
WRAP (Web Rephrase Augmented Pre-training)
WRAP (Web Rephrase Augmented Pre-training) is a synthetic-data pre-training method introduced in the paper "Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling," posted to arXiv on 29…