Training Set

A training set is the portion of data used to fit a statistical or machine learning model. Depending on the learning method, it may contain labeled input-output pairs, unlabeled observations, sequences of interaction, preference comparisons, or a mixture of these. The defining feature is functional: information from the training set is allowed to influence the learned model. Data reserved for model selection or final evaluation serve different purposes, even when all portions originated in the same collection.

The composition of a training set affects what a model can learn, where it is likely to work, and which failures an evaluation can reveal. A large set is not automatically a good one. Its relevance to the intended use, coverage of important cases, measurement and label quality, provenance, duplication, and relationship to evaluation data all matter. Dataset design is therefore part of model design, not merely a preliminary storage task.^[1][2]

Definition and role

In supervised learning, a training example normally consists of features and a target. The target may be a class, a number, a structured object, or a sequence. The learning algorithm uses the examples to choose model parameters that reduce a specified loss function. In self-supervised learning, targets are derived from the observations themselves, as when a language model predicts withheld or subsequent tokens. Unsupervised methods learn regularities without externally supplied targets. Reinforcement-learning systems instead learn from trajectories containing observations, actions, and reward or preference signals. These cases differ in format, but all use training data to alter the fitted system.

Training data should be distinguished from all data available to a project. Raw records may be excluded because they are outside scope, lack permission, fail quality checks, duplicate other records, or must be held out for evaluation. The resulting training set is therefore a designed sample, even when it began as an opportunistic collection.

A dataset can also change roles across experiments. A public benchmark's official training split may be used to fit one model, but a later researcher may partition that split again for development. Conversely, once feedback from a nominal test set guides feature engineering or model selection, that set has functionally become development data. The name of a file does not determine its statistical role; access and use do.

Many learning algorithms process the same set repeatedly. An epoch is one pass through the available training examples, although distributed sampling, streaming corpora, and examples generated during training can make the boundary approximate. Mini-batch optimization estimates an update from a subset at each step. Shuffling and sampling can change the sequence of updates without changing which records belong to the set.

Training, validation, and test data

A conventional predictive-modeling workflow separates three functions:

• The training set is used to estimate model parameters.
• The validation set, also called a development set, is used to compare model configurations, tune hyperparameters, choose thresholds, and make other development decisions.
• The test set is used to estimate the performance of the completed selection procedure on data that did not guide it.

This separation is intended to make evaluation informative about unseen cases. If the same observations determine both the fitted model and its reported performance, the estimate can be optimistically biased. The same problem can arise at the model-selection level: repeatedly choosing the best configuration on a finite validation set can overfit the selection criterion, even though no gradient update used those validation examples.^[3]

Repeated inspection of a holdout also makes it less independent of the development process. Dwork and colleagues formalized how adaptive reuse can overfit a holdout and proposed mechanisms for answering many adaptive queries with statistical guarantees.^[4] In ordinary projects, practical safeguards include limiting access to the final test set, recording which decisions used each evaluation, preregistering a primary analysis when appropriate, and obtaining a new evaluation sample after extensive adaptation. It is more accurate to treat test-set independence as a property to be managed than to claim that every test set can literally be used only once.

Split proportions

There is no universal 80/10/10 or 70/15/15 rule. A useful allocation depends on the amount of data, the precision required from validation and test estimates, the complexity of model selection, subgroup and rare-event requirements, and how closely future deployment can be sampled. Joseph derived allocation results for a particular linear-regression setting and explicitly contrasted them with rule-of-thumb ratios; the result does not establish one ratio for all machine-learning tasks.^[5]

On a very large independent sample, a small percentage may still produce a precise test estimate. On a small or imbalanced sample, a nominal percentage can leave too few cases of an important outcome to support a reliable conclusion. Confidence intervals or other uncertainty estimates should therefore be considered alongside point metrics. For high-stakes subgroup comparisons, the number and quality of examples in each relevant group can be more consequential than the overall split fraction.

When data are scarce, cross-validation can reuse observations across multiple train-validation partitions. It estimates performance by training separate models on multiple folds and evaluating each on the held-out fold. It does not allow a single fitted model to train on its own validation observations, and it does not by itself replace an external test set when extensive model selection has occurred. Nested cross-validation separates an inner selection loop from an outer evaluation loop and is one way to reduce selection bias.^[3][6]

Split strategy

Random splitting is appropriate only when exchangeability is a reasonable approximation and records do not have dependencies that cross the split. Other structures require different designs:

• Stratified splitting approximately preserves selected class proportions. It can stabilize comparisons when classes are uneven, but it does not correct sampling bias or guarantee enough examples for every subgroup.
• Group-aware splitting keeps related records, such as patients, users, households, documents from the same source, or repeated measurements of one subject, in a single partition. This prevents information about one group member from leaking through another.
• Temporal splitting trains on earlier observations and evaluates on later ones when the intended use predicts the future from the past. Randomly mixing time can expose the model to information that would not have existed at prediction time.
• Spatial, site-based, or domain-based splitting holds out locations, institutions, devices, or environments when generalization to a new domain is the actual question.

The correct design follows the intended claim. A random split answers how a procedure performs on a similar random draw from the sampled population. A future-time split asks about temporal transfer. A leave-site-out design asks about transfer to an unseen site. These are not interchangeable estimates.^[6][7]

Data leakage and contamination

Data leakage occurs when information unavailable at the intended prediction time, or information from an evaluation partition, influences training or model selection. Leakage can make an invalid system appear accurate. A survey by Kapoor and Narayanan identified leakage-related reproducibility failures across multiple scientific fields and organized them into a taxonomy extending from textbook split errors to open research problems.^[7]

Common forms include:

• fitting normalization, imputation, vocabulary, dimensionality-reduction, or feature-selection steps on the complete dataset before splitting;
• including a feature that is created after the outcome or is a proxy for it;
• placing duplicate or near-duplicate records in training and evaluation sets;
• splitting rows independently when several rows describe the same person, object, document, or event;
• selecting a model on the test set, including through repeated public-leaderboard submissions;
• pretraining on text or images that later appear in a benchmark, when the claim assumes benchmark independence.

The standard prevention pattern is to define the split before learning any data-dependent transformation. Preprocessing is fitted on training data and then applied without refitting to validation and test data. Pipeline abstractions can enforce this order. Resampling, feature selection, target encoding, and augmentation must also occur inside the training portion of each cross-validation fold, not before the folds are constructed.^[8]

Leakage checks should consider content rather than only record identifiers. Exact hashes find byte-identical duplicates, but near-duplicate documents, cropped images, translated passages, templated records, and shared source material may require similarity search or source-level grouping. The threshold and method should be documented because aggressive deduplication can also remove legitimate repeated phenomena.

For web-scale language-model corpora, train-evaluation overlap is a distinct contamination concern. Lee and colleagues found both repeated training text and validation text duplicated in the training corpora they studied. Their experiments showed that deduplication reduced memorized emissions and train-validation overlap in that setting.^[9] This does not imply that every duplicate is harmful: frequency may carry real information, and some tasks intentionally model recurrence. It does show why corpus builders should measure duplication and explain what they remove.

Constructing a training set

Training-set construction begins with an intended use, target population, unit of observation, and measurement protocol. Collectors need to decide what one example represents, how records are sampled, which time interval and locations are covered, and which exclusions are permitted. Those choices define the population to which evidence might generalize.

Data may be collected specifically for a task, drawn from administrative records, licensed from another party, contributed by users, generated by simulation, or assembled from public sources. Each route has different limitations. Administrative data reflect the process that created the records. Convenience samples can overrepresent people or conditions that are easy to observe. Web data inherit the selection, ranking, language, and availability patterns of the web and the collection pipeline. Simulations depend on the assumptions encoded by their generators.

Collection and filtering should be reproducible enough to audit. At a minimum, a dataset record should identify its source or source class, collection date or version, inclusion and exclusion criteria, transformations, deduplication method, known gaps, license or terms, and contact or governance process. Datasheets for datasets propose a structured set of questions covering motivation, composition, collection, preprocessing, distribution, maintenance, and uses.^[1] Data statements focus more specifically on the characteristics of language data and the populations represented.^[10] Google's Data Cards work adds an organizational framework for documenting datasets across producers, agents, and users.^[11]

Documentation is not proof that a set is suitable. It makes assumptions and limitations inspectable. The data-cascades study by Sambasivan and colleagues described how upstream data problems can compound through high-stakes machine-learning projects and how organizational incentives can obscure this work.^[2] A documented decision can still be wrong, but an undocumented pipeline is much harder to review or reproduce.

Labels, measurements, and disagreement

In a labeled training set, the target is a measurement, not an infallible fact. Labels can be entered incorrectly, defined ambiguously, inferred from proxies, or changed by context. A clinical code may reflect billing practice rather than a physiological state. A content-moderation label depends on a policy and sometimes on cultural or linguistic interpretation. A click is an observed action, not a direct measure of satisfaction.

Quality control can include annotator training, pilot rounds, adjudication, duplicate labeling, gold checks, and audits of samples and hard cases. Agreement statistics may be useful, but low agreement can indicate either poor instructions or genuine ambiguity. Collapsing all disagreement to one majority label discards information about uncertainty and population differences. The ChaosNLI project collected 100 annotations per example for selected natural-language-inference items and represented judgments as distributions, illustrating one alternative to a single "gold" answer.^[12]

Label-error detection can prioritize records for review, but algorithmic flags are not self-validating corrections. Northcutt and colleagues used confident learning followed by human review to study putative errors in ten benchmark test sets; they estimated a lower bound of 3.3 percent label errors on average across those particular datasets.^[13] The result is evidence that widely used evaluations can contain errors, not an estimate for all datasets. Changes to labels should retain provenance, the old value, the reason for change, and the review process.

Measurement quality also includes missingness, sensor calibration, extraction errors, unit consistency, timestamp accuracy, and transformations. Missing values are sometimes informative because the decision to measure a variable is itself part of a process. Imputation can be appropriate, but it should be learned within training data and tested under the missingness conditions expected in use.

Class imbalance and rare cases

A training set is imbalanced when some outcomes or groups occur much less often than others. Imbalance is not automatically an error. It may reflect the deployment distribution, and preserving it can be important for calibrated probability estimates. The problem is that an aggregate objective or metric may give rare but consequential cases too little influence.

Possible responses include collecting more relevant cases, weighting the loss, changing the sampling distribution used during optimization, adjusting a decision threshold, and reporting class-specific metrics. The appropriate choice depends on whether the goal is ranking, calibrated probabilities, equal error costs, or performance on a particular rare event.

Random oversampling repeats minority examples. Undersampling removes majority examples. Both can alter optimization, but neither creates new evidence about the underlying population. SMOTE constructs synthetic feature vectors between nearby minority examples and was introduced with experiments on several imbalanced classification datasets.^[14] It assumes that such interpolation is meaningful in the chosen feature space. Applying it to categorical, temporal, grouped, or highly structured data without an appropriate variant can create invalid examples. Resampling must happen after a split and within each training fold; otherwise synthetic or repeated information can leak into validation.

Evaluation should match the decision. Accuracy can conceal failure on a rare class. Precision-recall curves, per-class recall and precision, confusion matrices, calibration, cost-weighted measures, and uncertainty intervals may be more informative. Metric choice does not repair an unrepresentative sample, but it can prevent a dominant class from hiding errors.

Data augmentation

Data augmentation creates additional training examples by applying transformations intended to preserve, or deliberately alter in a controlled way, task-relevant meaning. Image crops, flips, color changes, audio perturbations, geometric transforms, text substitutions, and simulated variations are examples. The validity of a transformation is task-specific. A horizontal flip may preserve the label for many object categories but reverse meaning for text, road direction, anatomy, or handedness.

Augmentation acts as an inductive assumption about which variations a model should ignore or handle smoothly. It can improve generalization when that assumption matches the task, but it can introduce artifacts or erase information when it does not. Transform parameters should be chosen using training and validation data, and the unaugmented examples should remain traceable.

Some methods combine examples rather than transform one record. Mixup trains on convex combinations of pairs of inputs and their targets.^[15] CutMix replaces an image region with a patch from another image and mixes labels according to the patch area.^[16] Both are particular algorithms with empirical results, not universal recipes. Their assumptions fit some modalities and objectives better than others.

Augmentation differs from collecting independent observations. Ten transformations of one image do not provide the population coverage of ten newly sampled subjects. When train and evaluation records derive from a common original, all derivatives should be assigned to the same split.

Selecting and ordering examples

The set presented to an optimizer need not be a static random sample. Several learning paradigms decide which examples to label, include, weight, or present next.

Active learning uses a model or acquisition rule to select observations for labeling. A common goal is to obtain a useful model with fewer labeled examples than uniform sampling would require. Selection may be based on uncertainty, expected model change, diversity, or a combination. Settles' survey emphasizes that a learner also needs access to an appropriate pool or stream and an oracle capable of supplying labels.^[17] An uncertain case is not necessarily the most useful one: it can be an outlier, an annotation ambiguity, or a region irrelevant to deployment. The acquired sample is also no longer a simple random sample, so probability estimation and evaluation require care.

Semi-supervised learning combines labeled and unlabeled data. Many methods use consistency under perturbations, pseudo-labels, or representations learned from the unlabeled portion. FixMatch, for example, uses a model's high-confidence prediction on a weakly augmented input as a target for a strongly augmented version.^[18] Such methods rely on assumptions about how labeled and unlabeled distributions relate. If the unlabeled pool comes from a different population, confident pseudo-labels can reinforce errors rather than add reliable supervision.

Weak supervision derives approximate labels from rules, heuristics, external knowledge, or noisy sources. The data-programming framework models the accuracies and correlations of labeling functions so that they can be combined without requiring a hand label for every training example.^[19] Weak labels can expand coverage, but their sources, conflicts, estimated reliability, and failure modes remain part of the dataset's provenance.

Curriculum learning changes the order or distribution of examples over training, often progressing from easier or more prototypical cases to harder ones. Bengio and colleagues presented curriculum learning as a continuation strategy and reported experiments across several tasks.^[20] Later work includes self-paced and competence-based schedules. The term does not imply that every easy-to-hard ordering helps. Difficulty must be defined, and a schedule can delay rare, noisy, or safety-critical cases that the final system still needs to handle.

Hard-example mining does the converse in part: it gives more attention to examples on which a model currently performs poorly. This may focus capacity on decision boundaries, but it can also overweight mislabeled records. Review of repeatedly hard examples is therefore a data-quality tool as well as an optimization tactic.

Training-set size and learning curves

Adding relevant, independently informative examples often reduces estimation error, but the return is task- and model-dependent. A larger set can fail to help when new records duplicate old ones, come from the wrong population, contain noisy measurements, or exceed the model's ability to use the added variation. Conversely, a smaller curated set can outperform a much larger poorly matched one for a particular objective.

A learning curve plots performance against the amount of training data while holding the evaluation protocol fixed. It can reveal whether performance is still improving, whether variance across samples is large, or whether another bottleneck appears dominant. Comparisons should repeat sampling or fitting where feasible because one subset and one random seed can give a misleadingly smooth curve. Hoiem and colleagues propose a method for robustly estimating learning curves and use them to compare model-design choices in deep networks.^[21]

Scaling-law studies have found approximate power-law relationships among loss, model size, compute, and data within particular model families and regimes. Kaplan and colleagues reported empirical scaling relationships for autoregressive language models.^[22] Hoffmann and colleagues later trained a broad set of models and concluded that, under their assumptions and training setup, compute-optimal scaling required increasing training tokens and model size together more evenly than earlier practice.^[23] These are empirical regularities over defined ranges, not guarantees that arbitrary data improve a model or that one formula applies across modalities and distributions.

Data value is heterogeneous. An additional common example may contribute little, while a well-measured example from an uncovered condition may change performance materially. Selection based only on current-model loss can favor noise; selection based only on diversity can omit common high-density regions. Learning curves by subgroup, time period, source, and difficulty can expose these differences.

Dataset size should be reported in units meaningful to the modality: unique subjects and encounters as well as rows, hours as well as audio clips, documents as well as tokens, or trajectories as well as transitions. Counting after filtering and deduplication avoids presenting raw ingestion volume as effective training evidence.

Quality and fitness for purpose

Training-set quality is relational. The same set can be adequate for one task and unsuitable for another. Useful dimensions include:

• relevance, meaning that examples and targets correspond to the intended prediction or generation task;
• coverage, meaning that important conditions, outcomes, groups, and edge cases are represented well enough for the claim;
• measurement validity, meaning that features and targets measure the stated concepts;
• accuracy and consistency, including label, parsing, unit, and timestamp correctness;
• freshness, when the process or population changes over time;
• independence and duplication, relative to the intended estimator and evaluation;
• provenance and permission, including sources, transformations, licenses, consent where applicable, and restrictions;
• documentation and versioning, so that users can determine what was trained and reproduce or audit the pipeline.

These dimensions can conflict. Removing every unusual value may improve apparent consistency while deleting rare valid events. Deduplication may reduce memorization but distort naturally repeated frequency. Balancing groups can improve some comparisons while changing class priors used for probability calibration. Quality control therefore requires explicit objectives and preserved audit trails, not a single cleanliness score.

Automated checks can validate schemas, ranges, uniqueness constraints, file integrity, and distribution changes. Statistical tests can flag anomalies, but thresholds should be reviewed against domain knowledge. Human review is especially important where context determines whether an item is wrong, harmful, private, or out of scope.

Representativeness, bias, and distribution shift

A training set represents a population only with respect to specified variables and a sampling process. Demographic balance alone does not establish representativeness, and a sample can match marginal percentages while missing intersections, environments, behaviors, or measurement conditions that affect outcomes.

Dataset allocations across subgroups influence model performance. Rolf and colleagues formalized how group allocation, data collection, and learning objectives interact and showed that simply adding data is not equivalent to choosing allocations deliberately.^[24] Buolamwini and Gebru audited three commercial gender-classification systems using a benchmark balanced by skin type and gender. In their 2018 study, error rates were much higher for darker-skinned women than for lighter-skinned men.^[25] The study demonstrates why aggregate accuracy can conceal intersectional disparities; it does not establish a fixed property of all current systems.

Bias can enter through problem formulation, sampling, observation, labels, filtering, feature construction, missingness, and feedback from earlier systems. If a historical decision determines who receives an opportunity, using that decision as a target can reproduce the policy rather than measure underlying merit. If a sensor performs differently across environments, collecting equal numbers from each environment does not remove the measurement error.

Evaluation should be disaggregated where sample sizes and privacy allow. Subgroups should be chosen from the use context and known mechanisms of harm rather than searched indiscriminately until a difference appears. Reporting the sample count and uncertainty prevents tiny groups from producing overconfident comparisons. Interventions may include targeted collection, improved measurement, label-policy revision, reweighting, constraints, threshold changes, or changing the task itself.

Distribution shift means the relationship between training and use distributions changes. It may involve input frequencies, outcome prevalence, the relationship between inputs and outcomes, or the measurement process. A temporally later, geographically distinct, or institutionally independent test set can reveal specific forms of shift. Ovadia and colleagues compared predictive-uncertainty methods under several dataset shifts and found that uncertainty quality generally degraded as shift increased in their experiments.^[26]

A model can also be underspecified: many predictors have similar validation performance but behave differently in deployment-relevant conditions. D'Amour and colleagues documented this problem across several application domains and argued for stress tests aligned with known risks.^[27] More training data from the same narrow distribution may not resolve underspecification. Targeted data and evaluations that distinguish plausible solutions can be more useful.

Subbaswamy and Saria distinguish shift-stable relationships from unstable ones and propose causal approaches for transport across environments.^[28] Such methods depend on substantive assumptions about the data-generating process. They complement, rather than eliminate, the need to collect data from the conditions in which a system will operate.

Foundation-model training corpora

A large language model or other foundation model is often pretrained on a heterogeneous corpus rather than a conventional labeled table. Text models typically use self-supervised token-prediction objectives over mixtures of web pages, books, code, scholarly material, reference works, or licensed and curated collections. Vision, audio, and multimodal models use corresponding image, video, speech, caption, and paired corpora. The exact mixture is model-specific and may be only partly disclosed.

Web-scale construction is a pipeline, not a single download. Builders select snapshots and sources; extract content; identify language and document type; remove malformed, unsafe, private, or low-quality material under stated criteria; deduplicate; decontaminate evaluations; and choose mixture weights. Every step changes the effective distribution. A raw crawl such as Common Crawl is therefore not identical to the corpus on which a model trains.

Public research corpora illustrate different design choices. Dolma documents the sources, processing, and release of a three-trillion-token English corpus intended to support open language-model research.^[29] DataComp-LM provides a fixed raw pool, a standardized training framework, and downstream evaluations so researchers can compare data-curation strategies under controlled conditions.^[30] These projects are examples, not a complete inventory of commercial training data.

Filtering for apparent quality can improve a selected benchmark while narrowing language, style, or viewpoint. DataComp-LM's baseline experiments found model-based filtering effective in its controlled setting.^[30] That finding should not be generalized to every filter or social objective. A classifier trained to recognize a preferred source can inherit the source's demographics and genres. Mixture weights similarly encode a choice: upweighting code, mathematics, or a language can improve related capabilities while reducing the relative exposure to other material.

Deduplication operates at several levels. Exact duplicates can arise from mirrors and boilerplate. Near-duplicates include copied passages, templates, or versions with small edits. Cross-split decontamination searches for benchmark content or close variants. Lee and colleagues provide evidence that large language-model corpora and validation sets can contain substantial repeated text and that deduplication can reduce memorized output in the studied models.^[9] Search methods still have false positives and false negatives, so a corpus should state its unit, similarity rule, and treatment of naturally recurring text.

Instruction tuning adds demonstrations of desired responses or behavior after pretraining. Preference optimization uses comparisons, rankings, critiques, or rewards. These records are also training sets, even if they contain only a small fraction of pretraining's token count. Their label policy, annotator instructions, population, and quality can have disproportionate influence on observable behavior. A system's "training data" may therefore refer to several stages with different objectives and governance.

Corpus disclosure is sometimes limited by privacy, licensing, security, storage, or commercial considerations. A useful account can still report source categories, time ranges, languages, selection and exclusion rules, approximate mixture units, deduplication and decontamination methods, and known limitations. Claims about a particular model's undisclosed corpus should not be inferred from another model or from a generic crawl.

Synthetic training data

Synthetic data are generated by a simulator, procedural system, statistical model, or learned model rather than directly observed in the target environment. They can represent rare or dangerous events, vary factors under controlled conditions, protect some direct identifiers, or supply targets that are expensive to label. They can also carry exact ground truth from a simulator, such as object geometry or a known causal parameter.

Synthetic does not mean unbiased, private, or realistic. A simulator omits mechanisms its designers did not encode. A generative model can reproduce errors and imbalances from its own training data. Generated examples may be highly correlated and can leak memorized records. Privacy requires a threat model and empirical or formal safeguards; visual novelty is not a privacy guarantee.

Model-generated training data can support distillation, self-training, critique, or data augmentation. Its value depends on the generator, prompt or sampling process, filtering, and relationship to independent evidence. If a model trains repeatedly on data produced by models, errors can accumulate. Shumailov and colleagues analyze a recursive setting and provide theoretical and experimental evidence of "model collapse," in which learned distributions lose information about the original distribution.^[31] The result concerns specified recursive processes, not an inevitability for every use of synthetic data. Mixing generated examples with sufficiently representative real data, preserving provenance, and evaluating on independently collected data changes the setting.

Synthetic examples should be tagged with generator and version, parameters or prompts, random seed when relevant, postprocessing, filters, and parent records. Train-evaluation separation applies to the generator too: using a benchmark answer to produce a synthetic training example contaminates the benchmark even if the resulting wording differs.

Privacy, security, rights, and governance

Training data can expose information through the dataset itself, through model behavior, or through the surrounding pipeline. Removing obvious identifiers is not always sufficient because combinations of attributes can reidentify people or reveal sensitive facts. Access controls, minimization, retention limits, aggregation, and review of intended uses are relevant before model training begins.

Models can also reveal information statistically associated with their training records. Shokri and colleagues introduced membership-inference attacks that attempt to determine whether a record was in a model's training set.^[32] Carlini and colleagues demonstrated extraction of verbatim sequences from a language model in a controlled study, including sequences containing identifying information.^[33] Attack success depends on the model, data, access, and adversary, so these studies do not mean that every record is recoverable. They establish that model release is not automatically equivalent to keeping training data secret.

Differential privacy is a formal framework for limiting how much the output distribution of a computation can depend on one individual's record. Abadi and colleagues developed differentially private stochastic-gradient methods and an accounting approach for deep learning.^[34] A privacy guarantee must report its parameters, unit of protection, adjacency definition, and all relevant releases. Saying only that noise was added is not a differential-privacy claim. Privacy protection can also trade off with utility, especially for rare patterns, and does not resolve consent, intellectual-property, or representational questions.

Training pipelines have security risks. A poisoning attack modifies data or related training components to degrade a model or induce targeted behavior. NIST's 2025 adversarial-machine-learning taxonomy distinguishes availability, targeted, backdoor, model, and other poisoning attacks and reviews proposed mitigations.^[35] Controls can include trusted acquisition channels, source authentication, integrity hashes, provenance, anomaly review, robust training, access separation, and monitoring. No single filter rules out a determined attacker, especially when a pipeline intentionally ingests public or user-contributed material.

Rights and permissions depend on the source, jurisdiction, license, contractual terms, and use. A resource visible on the internet is not thereby free of copyright, privacy, confidentiality, database-right, or contractual constraints. Conversely, the legal treatment of training can differ across circumstances and jurisdictions. A general training-set article should not turn pending cases into universal rules. Dataset stewards should record the source and asserted basis for use, preserve licenses and notices, honor binding restrictions, and obtain qualified legal advice for a concrete project.

Consent is also contextual. A person may have agreed to one collection purpose without anticipating model training, release, or reuse in another domain. Public release can affect people who did not create the records but appear in them. Governance therefore includes escalation and removal processes, documentation of limitations, and decisions about whether some information should be excluded even when technically obtainable.

NIST's AI Risk Management Framework recommends maintaining training-data provenance and supporting attribution of system decisions to subsets of training data as aids to transparency and accountability.^[36] Provenance does not require that every project publish sensitive records. It can be maintained under controlled access while public documentation describes source classes, restrictions, and oversight.

Removal after training can be technically and organizationally difficult. Deleting a row from storage does not automatically remove its influence from existing model checkpoints, caches, replicas, or derived synthetic data. Projects should define retention, correction, opt-out, and retraining or machine unlearning procedures before they are needed, while avoiding claims that a particular unlearning method perfectly erases all influence unless that claim has been demonstrated for the system.

Versioning and reproducibility

A training set is a versioned artifact. Silent changes make experimental comparisons uninterpretable and can prevent incident investigation. Useful version records include:

• an immutable identifier or content manifest for the records;
• source versions and retrieval dates;
• code and configuration for extraction, filtering, joining, labeling, and splitting;
• schema and label-policy versions;
• counts before and after each major transformation;
• hashes or stable identifiers for split membership;
• known errors, corrections, removals, and superseded releases;
• the model runs and checkpoints trained from the version.

Large datasets may be represented by manifests rather than copied into a code repository. Tools such as DVC, object-store versioning, lakehouse snapshots, or content-addressed storage can help, but a tool does not define the governance policy. Raw data may need stricter access and shorter retention than derived features. Reproducibility metadata can be preserved without making restricted records public.

Pineau and colleagues' report on the NeurIPS 2019 reproducibility program describes a reproducibility checklist, code-submission policy, and community challenge designed to improve reporting and verification.^[37] For data, reproducibility means more than publishing a download link. A mutable URL can change, an API query can return new results, and a preprocessing dependency can alter output. A snapshot, manifest, environment, and executable pipeline make the claim more specific.

Dataset corrections should not be hidden to preserve a hash. A new version can document which records changed and why while the prior version remains identifiable or access-controlled as appropriate. Evaluations should say which version they used. When a correction changes a benchmark result, both the corrected analysis and the historical context may matter.

Practical review

Before training, a project can ask:

1. What system behavior is being learned, for which population, time period, and environment?
2. What is the unit of observation, and are units related by person, source, time, or location?
3. How were examples sampled, measured, labeled, filtered, and deduplicated?
4. Are important conditions and rare outcomes represented well enough for the intended claim?
5. Which transformations learn from data, and are they fitted only inside the training partition?
6. Does the split reproduce the deployment boundary, including groups, time, geography, and source?
7. Has exact and near-duplicate overlap with validation and test data been measured?
8. What label ambiguity, missingness, and measurement error remain?
9. Which privacy, security, consent, license, and retention constraints apply?
10. Can the exact training version, pipeline, split, and model run be reconstructed?
11. Which tests would detect drift, subgroup failure, contamination, or a poisoned source?
12. Which limitations are communicated to model developers, evaluators, deployers, and affected users?

No checklist makes a dataset universally valid. It creates reviewable claims. The central question is not whether a set is large or clean in the abstract, but whether the evidence produced by its collection and use supports the behavior and evaluation being claimed.

Simple example

Suppose a hospital wants to predict, at discharge, whether a patient will be readmitted within 30 days. A row-random split can leak information if the same patient has several admissions in different partitions. Features recorded after discharge would reveal future information. A label based only on readmissions to the same hospital may miss visits elsewhere. Data from one site and year may not support a claim about another site or a later policy regime.

A defensible design might group records by patient, train on earlier periods, reserve a later period or another site for evaluation, fit every imputation and encoding step within the training data, and report performance by relevant clinical and demographic groups with uncertainty. It would document the readmission definition, missing external events, exclusions, provenance, privacy controls, and dataset version. This example has no universal split ratio: the evaluation sample must be large and representative enough to answer the intended question.

References

1. ^Gebru, T., et al. "Datasheets for Datasets." *Communications of the ACM*, 2021. arxiv.org/...1803.09010
2. ^Sambasivan, N., et al. "Everyone Wants to Do the Model Work, Not the Data Work: Data Cascades in High-Stakes AI." *CHI*, 2021. research.google/...data-cascades-in-high-stakes-ai
3. ^Cawley, G. C., and Talbot, N. L. C. "On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation." *Journal of Machine Learning Research*, 2010. jmlr.org/...cawley10a
4. ^Dwork, C., et al. "Generalization in Adaptive Data Analysis and Holdout Reuse." *NeurIPS*, 2015. proceedings.neurips.cc/...8878a9d07405083-Abstract
5. ^Joseph, V. R. "Optimal Ratio for Data Splitting." *Statistical Analysis and Data Mining*, 2022. onlinelibrary.wiley.com/...sam.11583
6. ^scikit-learn developers. "Cross-validation: Evaluating Estimator Performance." *scikit-learn User Guide*. scikit-learn.org/...cross_validation
7. ^Kapoor, S., and Narayanan, A. "Leakage and the Reproducibility Crisis in Machine-learning-based Science." *Patterns*, 2023. arxiv.org/...2207.07048
8. ^scikit-learn developers. "Common Pitfalls and Recommended Practices." *scikit-learn User Guide*. scikit-learn.org/...common_pitfalls
9. ^Lee, K., et al. "Deduplicating Training Data Makes Language Models Better." *ACL*, 2022. aclanthology.org/2022.acl-long.577
10. ^Bender, E. M., and Friedman, B. "Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science." *Transactions of the ACL*, 2018. aclanthology.org/Q18-1041
11. ^Pushkarna, M., Zaldivar, A., and Kjartansson, O. "Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI." *FAccT*, 2022. research.google/...ocumentation-for-responsible-ai
12. ^Nie, Y., et al. "ChaosNLI: An Input Dataset for Better Understanding of Human Disagreement." *EMNLP*, 2020. aclanthology.org/2020.emnlp-main.734
13. ^Northcutt, C. G., Athalye, A., and Mueller, J. "Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks." *NeurIPS Datasets and Benchmarks*, 2021. datasets-benchmarks-proceedings.neurips.cc/...t-round1
14. ^Chawla, N. V., et al. "SMOTE: Synthetic Minority Over-sampling Technique." *Journal of Artificial Intelligence Research*, 2002. jair.org/...10302
15. ^Zhang, H., et al. "mixup: Beyond Empirical Risk Minimization." *ICLR*, 2018. openreview.net/forum
16. ^Yun, S., et al. "CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features." *ICCV*, 2019. openaccess.thecvf.com/..._Features_ICCV_2019_paper
17. ^Settles, B. "From Theories to Queries: Active Learning in Practice." *AISTATS Workshop on Active Learning and Experimental Design*, 2011. proceedings.mlr.press/...settles11a
18. ^Sohn, K., et al. "FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence." *NeurIPS*, 2020. research.google/...with-consistency-and-confidence
19. ^Ratner, A., et al. "Data Programming: Creating Large Training Sets, Quickly." *NeurIPS*, 2016. proceedings.neurips.cc/...ed5cea9f625f7ab-Abstract
20. ^Bengio, Y., et al. "Curriculum Learning." *ICML*, 2009. icml.cc/...119.pdf
21. ^Hoiem, D., et al. "Learning Curves for Analysis of Deep Networks." *ICML*, 2021. proceedings.mlr.press/...hoiem21a
22. ^Kaplan, J., et al. "Scaling Laws for Neural Language Models." 2020. arxiv.org/...2001.08361
23. ^Hoffmann, J., et al. "Training Compute-Optimal Large Language Models." *NeurIPS*, 2022. deepmind.google/...l-large-language-model-training
24. ^Rolf, E., et al. "Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data." *ICML*, 2021. proceedings.mlr.press/...rolf21a
25. ^Buolamwini, J., and Gebru, T. "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification." *FAT*, 2018. proceedings.mlr.press/...buolamwini18a
26. ^Ovadia, Y., et al. "Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift." *NeurIPS*, 2019. proceedings.neurips.cc/...371888657d2eb1d-Abstract
27. ^D'Amour, A., et al. "Underspecification Presents Challenges for Credibility in Modern Machine Learning." *Journal of Machine Learning Research*, 2022. jmlr.org/...20-1335
28. ^Subbaswamy, A., and Saria, S. "From Development to Deployment: Dataset Shift, Causality, and Shift-Stable Models in Health AI." *Biostatistics*, 2020. academic.oup.com/...5631849
29. ^Soldaini, L., et al. "Dolma: An Open Corpus of Three Trillion Tokens for Language Model Pretraining Research." *ACL*, 2024. aclanthology.org/2024.acl-long.840
30. ^Li, J., et al. "DataComp-LM: In Search of the Next Generation of Training Sets for Language Models." *NeurIPS Datasets and Benchmarks*, 2024. proceedings.neurips.cc/...ets_and_Benchmarks_Track
31. ^Shumailov, I., et al. "AI Models Collapse When Trained on Recursively Generated Data." *Nature*, 2024. nature.com/...s41586-024-07566-y
32. ^Shokri, R., et al. "Membership Inference Attacks Against Machine Learning Models." *IEEE Symposium on Security and Privacy*, 2017. ieee-security.org/...program-papers
33. ^Carlini, N., et al. "Extracting Training Data from Large Language Models." *USENIX Security*, 2021. usenix.org/...carlini-extracting
34. ^Abadi, M., et al. "Deep Learning with Differential Privacy." *ACM CCS*, 2016. research.google/...rning-with-differential-privacy
35. ^Vassilev, A., et al. *Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations*. NIST AI 100-2e2025, 2025. csrc.nist.gov/...final
36. ^Tabassi, E. *Artificial Intelligence Risk Management Framework (AI RMF 1.0)*. NIST AI 100-1, 2023. nvlpubs.nist.gov/...NIST.AI.100-1.pdf
37. ^Pineau, J., et al. "Improving Reproducibility in Machine Learning Research." *Journal of Machine Learning Research*, 2021. jmlr.org/...20-303