Data & Datasets
113 AI Wiki articles on Data & Datasets. The most referenced are Training Set, Synthetic data and Feature Engineering.
113 articlesRSS
Showing 1-60 of 113 articles
Amazon Mechanical Turk
Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace operated by Amazon in which businesses and researchers ("requesters") post small paid tasks that...
AI EthicsAI History
BookCorpus
BookCorpus (also written BooksCorpus, and sometimes called the Toronto Book Corpus) is a text dataset built from free, self-published English-language ebooks...
AI EthicsNatural Language Processing
Bucketing
Bucketing, also called binning or discretization, is a feature engineering technique in machine learning that converts a continuous feature into a small number...
Machine Learning
C4 (Colossal Clean Crawled Corpus)
C4 (Colossal Clean Crawled Corpus) is a roughly 750 GB, cleaned, English-language web text dataset of about 365 million documents and 156 billion tokens that...
Natural Language Processing
CIFAR-10
CIFAR-10 is a labeled dataset of 60,000 small color images sorted into 10 mutually exclusive object categories, with 6,000 images per class, used as a standard...
AI BenchmarksComputer Vision
COCO dataset
COCO (Common Objects in Context) is a large-scale dataset for object detection, image segmentation, keypoint detection, and image captioning. Created by a team...
Computer VisionMachine Learning
Categorical Data
Categorical data, also called qualitative data, is data whose values are discrete labels or groups (such as colors, country names, or blood types) rather than...
Machine LearningStatistics
CharXiv
CharXiv Charting Gaps in Realistic Chart Understanding in Multimodal LLMs Abbreviation An evaluation suite for assessing chart understanding capabilities...
AI Benchmarks
Class-Imbalanced Dataset
A class-imbalanced dataset is a dataset in which the distribution of examples across the target classes is significantly unequal, so that one class (the...
Machine Learning
Common Corpus
Common Corpus is the largest fully open, multilingual dataset for pretraining large language models, assembled and released by the French AI research lab...
Natural Language ProcessingOpen Source AI
Common Crawl
Common Crawl is a nonprofit 501(c)(3) organization that maintains a free, open repository of web crawl data, and it is the single largest publicly available...
Machine LearningNatural Language Processing
Common Pile
Common Pile v0.1 is an 8 terabyte corpus of openly licensed and public domain text, released on June 5, 2025, by EleutherAI and a consortium of more than two...
Natural Language ProcessingOpen Source AI
Continuous Feature
A continuous feature is a numeric input variable in machine learning and statistics that can take any value within a range, including decimals and fractions,...
Machine LearningStatistics
Convenience Sampling
Convenience sampling (also called grab sampling, accidental sampling, or opportunity sampling) is a non-probability sampling method in which data points or...
Machine LearningStatistics
Cosmopedia
Cosmopedia is an open synthetic pretraining dataset released by Hugging Face in February 2024, made up of textbooks, blog posts, stories, and WikiHow-style...
Large Language ModelsOpen Source AI
Coverage Bias
Coverage bias is a type of selection bias that occurs when the method used to collect data systematically excludes part of the target population, so the sample...
AI EthicsMachine Learning
DCLM (DataComp for Language Models)
DCLM, short for DataComp for Language Models (also styled DataComp-LM), is an open benchmark, dataset, and software framework, released in June 2024, for...
AI BenchmarksNatural Language Processing
Data Augmentation
See also: Machine learning terms Data augmentation is a set of techniques that artificially expand the size and diversity of a training dataset by applying...
Deep LearningMachine Learning
Data Provenance Initiative
The Data Provenance Initiative (DPI) is a volunteer-led, multi-institution research collective that audits and documents the licenses, sources, creators, and...
Machine Learning
Data Set or Dataset
See also: Machine learning, Training set, Validation set, Test set A dataset (also written as "data set") is a structured collection of data points used to...
Machine Learning
Data contamination
Data contamination is the presence of benchmark evaluation data in the corpus used to train a model, so that reported scores partly measure recall of material...
AI BenchmarksModel Evaluation
Data preprocessing
Data preprocessing is the set of operations applied to raw data to clean and transform it into a form a machine learning model can use, covering deduplication,...
Machine Learning
Data-centric AI (DCAI)
Data-centric AI (DCAI) is the discipline of systematically engineering and improving the data used to train a machine learning model, rather than holding the...
MLOps
DatologyAI
DatologyAI is a Redwood City, California artificial-intelligence startup that builds automated tools for curating, deduplicating, and composing the training...
AI Companies
Dense Feature
A dense feature is a feature in machine learning whose vector representation consists mostly or entirely of non-zero values, typically stored as a dense...
Machine Learning
Derived label
See also: Machine learning terms A derived label is a label that has been generated programmatically or inferred from other observable signals, rather than...
Machine Learning
Dimension Reduction
See also: Machine learning terms Dimensionality reduction, also known as dimension reduction, is the process of transforming data from a high-dimensional space...
Machine Learning
Discrete Feature
A discrete feature is a feature (a variable in a dataset) that takes one of a finite or countably infinite set of distinct values, such as a category or an...
Machine Learning
Distribution shift
Distribution shift is the condition in which the probability distribution that produced a model's training data differs from the distribution that produces the...
AI SafetyMachine Learning
Dolma
Dolma is an open three-trillion-token English pretraining corpus released by the Allen Institute for AI (AI2) to power its fully open OLMo language models and...
Large Language ModelsOpen Source AI
Downsampling
See also: Machine learning terms Downsampling is the process of reducing the number of samples, the spatial resolution, or the number of data instances in a...
Deep LearningMachine Learning
Ego-Exo4D
Ego-Exo4D is a large-scale, multimodal, multiview video dataset and benchmark suite for computer vision research on skilled human activity, a central resource...
Computer VisionMeta AI
Ego4D
Ego4D is a large-scale egocentric (first-person) video dataset and benchmark suite for computer vision, assembled by Meta AI (then Facebook AI Research)...
Computer VisionMeta AI
Feature
In machine learning and statistics, a feature is an individual measurable property or characteristic of a phenomenon being observed, used as an input variable...
Machine Learning
Feature Cross
See also: Machine learning terms, Feature engineering A feature cross (also called a crossed feature or feature interaction) is a synthetic feature created by...
Machine Learning
Feature Engineering
Feature engineering is the process of using domain knowledge to create, transform, and select features from raw data so that machine learning models can learn...
Machine Learning
Feature Extraction
See also: Machine learning terms, Feature engineering, Dimension reduction Feature extraction is the process of transforming raw data into a smaller set of...
Machine Learning
Feature Selection
Feature selection is the process of choosing a subset of the most relevant input variables (features) from a larger candidate pool for use in a machine...
AlgorithmsMachine Learning
Feature Set
See also: Feature, Feature Engineering, Feature Extraction, Feature Vector A feature set is the complete collection of input variables (features, attributes,...
Data ScienceMachine Learning
Feature Vector
A feature vector is an n-dimensional, ordered list of numerical values that represents the measurable properties of an object, data point, or observation in a...
Machine Learning
FineWeb
FineWeb is a large-scale, open pretraining dataset for large language models (LLMs) created by Hugging Face. Released in April 2024, it contains approximately...
Machine LearningNatural Language Processing
FineWeb-2
FineWeb-2 (also written FineWeb2) is a massively multilingual web pretraining dataset released by Hugging Face in December 2024. It is the multilingual...
Machine Learning
FineWeb-Edu
FineWeb-Edu is an open, English-language pretraining dataset of roughly 1.3 trillion tokens, built by filtering the much larger FineWeb web corpus down to the...
Large Language ModelsMachine Learning
Ground Truth
Ground truth is verified, correct information that serves as the authoritative reference for training and evaluating machine learning models. In supervised...
Machine Learning
HotpotQA
HotpotQA is a large-scale, multi-hop question answering dataset of about 112,779 crowd-authored question-and-answer pairs over English Wikipedia, whose answers...
AI BenchmarksArtificial Intelligence
How to Prevent OpenAI and Google From Training Their LLMs on Your Website's Data
See also: Guides You can stop OpenAI, Google, Anthropic, and most other major AI companies from using your website to train large language models (LLMs) by...
Large Language Models
Imbalanced Dataset
An imbalanced dataset is a dataset used in machine learning where the classification categories are not approximately equally represented, so that one class...
Machine Learning
Instance
In machine learning and statistics, an instance is a single data point in a dataset: the values of one row of features and, optionally, a label [18]. It is the...
Machine Learning
Inter-rater agreement
See also: Machine learning terms Inter-rater agreement is the degree of consensus among two or more independent raters when they label or score the same set of...
Model EvaluationStatistics
Iris dataset
The Iris dataset, sometimes referred to as Fisher's Iris dataset or the Iris flower dataset, is a multivariate dataset introduced by the British statistician...
AI BenchmarksMachine Learning
LAION
LAION (Large-scale Artificial Intelligence Open Network) is a German non-profit organization, registered as LAION e.V. in Hamburg, that builds and releases...
Computer VisionMachine Learning
LAION-5B
LAION-5B is an open dataset of approximately 5.85 billion CLIP-filtered image and text pairs scraped from the public internet, released by LAION (Large-scale...
Generative AI
LVIS (Large Vocabulary Instance Segmentation)
LVIS (Large Vocabulary Instance Segmentation, pronounced "el-vis") is a large-scale instance segmentation benchmark for computer vision that targets the...
Computer Vision
Label
See also: Machine learning terms In machine learning, a label is the target output value associated with a single training example: the correct answer that a...
Machine Learning
Low-Resource Languages
Low-resource languages, in natural language processing (NLP), are languages with little of the raw material that modern language technology depends on:...
Machine LearningNatural Language Processing
MMMLU
Multilingual Massive Multitask Language Understanding Abbreviation Professional human translations of the MMLU test set into 14 languages, released by...
AI Benchmarks
MNIST
The Modified National Institute of Standards and Technology (MNIST) database is a collection of 70,000 grayscale images of handwritten digits (0 through 9)...
Computer VisionMachine Learning
Mecka
Mecka (Mecka AI) is a robotics data company that builds large-scale human-motion datasets used to train robots. The startup records how people move, walk, and...
Robotics Companies
MetaCLIP
MetaCLIP (Metadata-Curated Language-Image Pre-training) is a data curation recipe and a family of vision-language models from Meta AI, introduced in the 2023...
Meta AIMultimodal AI
MimicGen
MimicGen is a data generation system developed by researchers at NVIDIA's Seattle Robotics Lab and Learning and Perception Research group that automatically...
Embodied AINVIDIA