Model

A model in machine learning is a representation used to map inputs to outputs, assign probabilities, describe relationships, or choose actions. In the broader language of artificial intelligence, a model can be learned from data or constructed from rules and knowledge. The OECD describes an AI model as a core component of an AI system that is used to infer outputs from inputs, while the complete system may also include data collection, software, sensors, user interfaces, policies, and human decision makers.^[1] The word therefore names neither every part of an AI application nor only a file of neural-network weights.

In ordinary ML usage, "the model" usually means a particular trained instance: a decision function, probability distribution, tree, ensemble, neural network with learned state, or another fitted object. Its behavior depends on more than a family name. Training data, preprocessing, objective, hyperparameters, random choices, learned state, software semantics, and the way an input is presented can all distinguish two instances. At use time, inference applies that instance to new input. A hosted product or service may wrap one or more models and can change routing, prompts, tools, safety rules, retrieval data, or model versions without changing its public name.

This article focuses on the AI and ML meaning of model. Statistical and scientific models have wider traditions in which a model may primarily explain a data-generating process or represent a physical phenomenon. Model architecture, parameters, checkpoints, deployable packages, and AI systems are related but not interchangeable concepts.

Scope and meaning

"Model" is an overloaded word. In science, it can mean a simplified representation of a phenomenon. In statistics, it can mean a family of possible probability distributions together with assumptions about how observations arise. In engineering, it may be a physical, mathematical, or logical representation used for analysis or control. ML inherits all of these senses but often emphasizes a fitted input-output object produced by a learning procedure.

The broad definition matters because not every AI model is a neural network and not every model is learned in the same way. A manually written rule base, a learned decision tree, a nearest-neighbor predictor that retains examples, a probabilistic graphical model, and a neural network can all serve as models. The OECD account explicitly includes manually and automatically built models, statistical models, decision trees, neural networks, and models of an environment.^[1] Even "model-free" reinforcement learning uses the word in a narrower sense: it lacks an explicit transition model of the environment, but its learned policy or value function can still be an AI model in the general input-output sense.^[1]

ML also sits between two modeling traditions. Breiman distinguished approaches that begin with an explicit stochastic account of how data are generated from approaches that emphasize algorithms capable of accurate prediction.^[4] The distinction is useful, but it is not a clean division between statistics and machine learning. A modern model can be both probabilistic and highly predictive, and a prediction system can still be studied for causal, scientific, or explanatory purposes. The intended question determines which assumptions and evidence matter.

For a learned model, a compact abstraction is:

M = A(D, lambda, r)

Here A is a learning procedure, D is the training data and its representation, lambda denotes settings chosen outside the fitting run, and r captures random choices. The output M is one fitted model instance. This notation makes two points visible. The learning algorithm is not the model it produces, and changing data, settings, software behavior, or randomness can produce a different model even when the family name is unchanged. Bergstra and Bengio similarly formalize a learning algorithm as mapping a training set to a learned function while hyperparameters configure that algorithm.^[3]

The abstraction is intentionally wider than f(x; theta). Many neural and linear models can be written as a function with a parameter vector theta, but other models store split rules, support vectors, prototypes, exemplars, probability tables, symbolic expressions, or collections of component models. Calling every model "an architecture plus weights" erases these cases.

Model, algorithm, architecture, and state

Several neighboring terms are often collapsed into "model." Keeping them separate makes technical claims testable.

Scroll sideways for more →

Architecture is a constraint on possible models, not generally a unique model. Two neural networks can share the same layer layout and still be different because their learned values differ. Two decision trees trained with the same code can differ in their split variables, thresholds, leaves, or tie-breaking. A family such as linear regression denotes many possible fitted functions, not one artifact.

Parameters and weights are also not perfect synonyms. "Weight" commonly denotes a numeric coefficient learned by a neural network or linear model. "Parameter" is broader and can include biases, distribution parameters, leaf values, or other learned quantities. Some runtime state affects output without being treated as a trainable weight, such as normalization statistics or a vocabulary mapping. Conversely, a mathematical parameter may be fixed rather than estimated in a particular use. The meaning should be stated for the model family at hand.

Hyperparameters are relative to a training procedure. A regularization coefficient selected before one training run is a hyperparameter of that run. If an outer optimization procedure searches over that coefficient, it is an output of the outer procedure but remains a configuration of each inner learner.^[3] The data split, feature construction, stopping rule, and random seed can be just as important to the resulting instance even though teams do not always label them hyperparameters.

The term "state" is often safest when the representation is mixed. An ensemble may store several fitted components and aggregation rules. A tokenizer may define how text becomes indices. A tree stores a topology and node values. A probabilistic model may store parameters and a specified factorization. Whether preprocessing is considered part of the model or part of the pipeline is a packaging choice, but that boundary must remain consistent when the object is evaluated or deployed.

How a learned model is formed

Model training estimates, constructs, or updates model state from examples or interaction. In a supervised problem, a common formulation chooses parameters to reduce average loss:

J(theta) = (1 / n) sum_i L(f(x_i; theta), y_i)

The expression contains a model function f, training examples (x_i, y_i), a loss function L, and an optimization procedure. It is a useful pattern, not a universal definition of learning. Tree induction, nearest-neighbor storage, Bayesian updating, rule induction, clustering, search, and evolutionary procedures may build useful models without following differentiable gradient optimization.

Training is indirect. What users care about is behavior on relevant future cases, but the available calculation usually uses a finite training sample and a tractable objective. Goodfellow, Bengio, and Courville distinguish the empirical training objective from expected risk under the underlying data distribution, and they note that a surrogate loss is often optimized because the desired measure is not directly tractable.^[2] Reducing training loss is therefore evidence about a procedure, not proof that the model meets its deployment objective.

Different learning settings change what information guides construction:

• Supervised learning uses examples paired with targets, labels, outcomes, or structured responses.
• Unsupervised learning seeks structure without those target labels, for example clusters, latent representations, or a density model.
• Reinforcement learning updates policies, values, or environment models through observations, actions, and reward signals.

These categories can be combined, and the same artifact can pass through multiple stages. A representation may be learned without task labels, adapted with labeled examples, and later updated from preference or reward data. What matters for identity is the actual sequence, data, objectives, and state transitions, not a single label attached to the final artifact.

Randomness can enter through initialization, example order, data augmentation, sampling, hardware kernels, or distributed execution. The learning procedure therefore defines a distribution over possible fitted models unless every relevant choice and operation is fixed. Two outputs can have similar aggregate scores and still make different predictions on particular cases. Treating the seed as part of the training record helps investigation, but it does not by itself guarantee identical results across software and hardware.

Training also creates a path, not only a final point. Intermediate snapshots can support recovery, comparison, or selection. A stopping decision chooses one point on that path, frequently using validation behavior rather than the lowest training objective. The selected state is a model instance. Optimizer buffers, gradient-scaler state, data-loader position, and scheduling state may be necessary to continue the run but are not ordinarily needed to perform inference.

Forms of model behavior

Models can be classified along several independent axes. None is a complete taxonomy.

A deterministic model returns the same output for the same fully specified input and state under fixed execution semantics. A probabilistic model represents uncertainty or a distribution, such as p(y | x) or p(x). Sampling from a probabilistic model can produce different outputs without the stored parameters changing. A nominally deterministic function can also vary in practice because the surrounding system changes preprocessing, retrieval results, decoding settings, or numerical execution.

A discriminative model directly estimates a boundary, score, or conditional relationship useful for distinguishing outputs. A generative model represents or approximates how observations or structured outputs may be generated and can often be sampled. These descriptions overlap in modern systems. A model may contain discriminative components inside a generative pipeline, and a generative representation can be used for classification or ranking.

Parametric models describe their fitted state using a fixed finite set of parameters for a chosen structure. Nonparametric methods allow effective complexity or stored state to grow with data. The names do not mean "has no parameters" or "makes no assumptions." Both kinds embody choices about representation, similarity, smoothness, priors, or allowable functions.

An ensemble combines multiple fitted models or training outcomes. The ensemble itself is a model if its component states and combination rule jointly determine an output. Reporting only the architecture or family of one component would not identify it.

Models can also be static or adaptive in operation. A static deployment keeps model state fixed until an explicit replacement. An adaptive system updates state from new observations or feedback. Adaptation can improve relevance, but it also changes the object being evaluated. Controls must specify what is allowed to update, how new versions are tested, how harmful feedback loops are limited, and whether earlier behavior can be reconstructed. Calling a service "the same model" while its state continuously changes is an operational convention, not literal artifact identity.

Finally, input-output role does not determine internal form. A classifier might be a linear function, tree, nearest-neighbor rule, neural network, probabilistic program, or ensemble. A model of environment dynamics can support planning rather than directly emit a user-facing prediction. A large model can support many tasks through context or adaptation, while a small model can be tightly specialized. Size, family, task, and learning setting should be described separately.

Evaluation, selection, and generalization

Model evaluation asks how a specified model behaves under a specified procedure. A metric without its dataset, sampling unit, preprocessing, threshold, subgroup definition, uncertainty estimate, and comparison rule is incomplete. The same model can be strong under one error cost or operating point and unacceptable under another.

The central goal is generalization: useful behavior beyond the examples used to fit the model. It is not a context-free scalar property. Generalization is always relative to a target population, environment, task, and loss. A model can generalize from one random sample of a hospital's records to another while failing across hospitals, devices, years, or patient groups.

Capacity helps explain two common failures. A model with too little effective model capacity may show underfitting, missing relevant structure in both training and evaluation data. A flexible model can show overfitting, matching peculiarities of the training sample that do not persist. The behavior depends on the relationship among model family, data, regularization, optimization, and evaluation design. Parameter count alone is not a universal measure of usable capacity.

Data are commonly divided by role. Training data affect fitted state. Validation data guide hyperparameters, thresholds, stopping, architecture choices, or candidate selection. Test data estimate performance after those decisions. Repeatedly choosing models by test results turns the test set into a selection instrument and weakens its role as independent evidence. Cross-validation can estimate variability and use limited data efficiently, but it does not remove the need to match splits to groups, time, geography, or other dependence in the intended setting.

Selection can overfit even when every individual training run is sound. Cawley and Talbot show that variance in a model-selection criterion can be exploited by the selection process, creating optimistic bias in subsequent performance estimates.^[5] Comparing many pipelines, seeds, feature sets, or checkpoints expands the effective search. The final report should account for the whole selection procedure, not present the winning candidate as if it had been chosen in advance.

Accuracy and uncertainty are different questions. Calibration measures whether stated confidence corresponds to observed frequencies under a defined evaluation setting. Guo and colleagues found that modern neural networks can be accurate yet poorly calibrated and evaluated post-hoc temperature scaling as one corrective method.^[6] Calibration can change under a new population or after thresholding, and a single aggregate curve can hide subgroup differences.

In-distribution success can leave behavior underspecified. D'Amour and colleagues describe pipelines that admit many predictors with similarly strong held-out performance but materially different behavior under stress or deployment conditions.^[7] Evaluating only the criterion used for selection cannot reveal every relevant difference among those predictors. Stress tests, subgroup analysis, perturbation studies, alternative datasets, and domain-specific error review supply different evidence.

Distribution shift makes the target itself a moving or different environment. The WILDS benchmark assembled real-world shifts across domains and found that standard training often produced substantially lower out-of-distribution than in-distribution performance.^[8] A shift result is not proof that every deployment will fail, but it shows why a random held-out split is not sufficient when the operational population differs in known ways. Monitoring inputs and outcomes can detect some changes, yet labels may arrive late and unobserved shifts may remain.

No evaluation makes a model universally "good." Evidence supports a bounded claim: this exact version, executed in this pipeline, met stated criteria on these data under these conditions. Deployment decisions must also consider costs of error, human fallback, latency, resource use, privacy, robustness, accessibility, and consequences for affected people.

From model object to deployed system

A fitted model becomes useful through software that obtains inputs, validates and transforms them, executes the model, interprets outputs, and takes or recommends an action. Sculley and colleagues describe the model code as only a small part of a production ML system and identify risks from data dependencies, feedback loops, configuration, undeclared consumers, and changes in the external world.^[10] A model score measured in isolation therefore does not establish system performance.

The boundary can be seen in a simple prediction service:

1. An input contract defines accepted fields, units, missing-value behavior, and authorization.
2. Feature or prompt construction turns the request into the model's expected representation.
3. A runtime executes one or more models.
4. Postprocessing may calibrate scores, select a threshold, filter content, or combine model output with rules.
5. Application logic decides what to display, store, route, or act upon.
6. Observability, review, rollback, and human procedures manage operation over time.

Only some of those steps belong to the model, and teams may draw the packaging boundary differently. The critical requirement is that evaluation and deployment use compatible boundaries. A model evaluated with one tokenizer, feature mapping, or threshold is not fully represented by weights that are later paired with another.

A checkpoint is one kind of artifact. It may include learned model state, optimizer state, training progress, random-number state, or other information needed to resume a run. PyTorch documentation, for example, distinguishes a model state_dict from a general checkpoint that also stores optimizer and epoch state, and it requires the model structure to be instantiated before a state dictionary is loaded.^[13] That framework-specific example illustrates a general rule: numeric values need semantics and structure.

An interchange or deployment format can encode more of those semantics. The ONNX intermediate-representation specification defines a model as a versioned graph with operator-set imports, values, initializers, and metadata; it separately defines training information and runtime requirements.^[12] Serialization still does not guarantee identical execution everywhere. Operator versions, unsupported operations, shapes, numerical precision, external data, preprocessing, and runtime implementation can affect compatibility and output.

Model deployment adds infrastructure, access control, scaling, observability, and release management. A public API name may route among model versions, invoke tools, retrieve changing data, or apply undisclosed postprocessing. It is more accurate to call that object a service or system unless the provider identifies an exact underlying model artifact. MLOps practices coordinate data, training, registry, testing, release, monitoring, and rollback, but automation does not remove the need to define what constitutes a version.

Identity, versions, and reproducibility

A useful model identifier should resolve to enough information to reconstruct or at least distinguish the relevant behavior. Depending on the use, that can include:

• architecture or executable graph and its version;
• learned state and an artifact hash;
• feature schema, tokenizer, preprocessing, and postprocessing;
• training data provenance, snapshot, filters, and split logic;
• objective, regularization, hyperparameters, stopping, and selection rules;
• source code, dependencies, compiler or runtime, and operator versions;
• random seeds and nondeterminism settings;
• precision, device, driver, and distributed configuration;
• intended task, input contract, thresholds, and evaluation record.

Not every field belongs inside a single file. A registry can link immutable artifacts to signed metadata, code revisions, datasets, and reports. The important property is referential clarity. A mutable filename such as latest.bin or a marketing label such as "pro" is not enough for an auditable experiment.

Several changes deserve a new version even if the family name stays the same. Retraining on revised data creates new learned state. Changing a tokenizer changes the function from user input to model input. Updating a numerical runtime can alter edge-case behavior. Modifying a decision threshold changes system decisions even when the score-producing model is untouched. Whether the threshold is called part of the model or part of the application, the evaluated bundle must be versioned.

Reproducibility is also weaker than identity. Pineau and colleagues define reproducibility in ML research as obtaining similar results with the same code and data, not necessarily producing bit-for-bit identical artifacts.^[11] A result can reproduce statistically while individual fitted parameters differ. Conversely, loading the same bytes does not guarantee the same externally observed behavior if code, operators, hardware, preprocessing, decoding, or dependencies change.

Hashes are useful for integrity and exact-byte identity, but two files with different container metadata can encode equivalent model state, while two files with the same architecture label can encode very different behavior. Functional comparison can test outputs over a defined input set, but passing that set is not proof of equivalence on every possible input. Reproducibility records, integrity hashes, and behavioral tests answer different questions and should be kept together.

An adaptive system requires an additional record of time and state transitions. If updates occur online, the model used for one decision may not be the model used for the next. Logs need to identify which state, context, and surrounding system produced a material output, subject to privacy and security constraints. Rollback requires a recoverable artifact and compatible dependencies, not only a previous model name.

Documentation, governance, and security

A model needs documentation proportionate to its impact. The model card proposal calls for intended uses, evaluation procedures, performance characteristics, relevant conditions, and results across groups important to the application.^[9] A useful record can also include excluded uses, training-data provenance at an appropriate level, limitations, ethical and legal considerations, security assumptions, version history, licenses, and contact or maintenance responsibility.

Documentation should separate observation from inference. "The model scored 0.91 on dataset X under procedure Y" is an observed evaluation result. "The model is safe," "fair," "robust," or "ready for healthcare" is a much broader judgment that requires contextual evidence. Explainable AI methods can help inspect associations, errors, or decision factors, but an explanation method does not automatically establish causal validity, fairness, or suitability.

The NIST AI Risk Management Framework treats risk management as a lifecycle and contextual activity organized around govern, map, measure, and manage functions.^[14] This system-level framing is important. Some risks originate in model behavior, while others arise from data collection, user interface, automation, access, incentives, or the absence of human recourse. A lower model error rate does not by itself resolve those surrounding risks.

Security must likewise cover more than file access. NIST's adversarial-ML taxonomy organizes attacks by lifecycle stage, attacker goals, capabilities, and knowledge, and includes poisoning, evasion, privacy, and misuse concerns across predictive and generative systems.^[15] Threats can target training data, learned state, an inference interface, feedback channels, or downstream integrations. Relevant controls include provenance, access control, isolation, input and output validation, rate limits, monitoring, incident response, and testing against a stated threat model.

Model artifacts should be treated as software supply-chain objects. A loader may interpret executable code or a graph of operations, dependencies can carry vulnerabilities, and an apparently data-only format can still induce excessive resource use or malformed shapes. Producers should publish integrity information and consumers should obtain artifacts from authenticated sources, inspect format-specific loading behavior, constrain resources, and test in an isolated environment when trust is limited.

Governance assigns responsibility for approving data, training, evaluation, release, monitoring, change, and retirement. It should also specify who can accept residual risk and how affected users can challenge material outcomes. These controls attach to an actual use, not to the abstract model family. The same fitted object can be low consequence in a private experiment and high consequence when used to allocate benefits or control physical equipment.

Pretrained, adapted, and foundation models

A pre-trained model is a fitted model whose learned state is reused as a starting point or component for another task. Reuse may freeze the original state, add a task-specific head, learn a small adapter, or update some or all parameters. Experiments by Yosinski and colleagues showed that transferability in a convolutional network varied by layer and by distance between source and target tasks.^[17] The broader lesson is durable: prior training can help, but reuse does not guarantee suitability for a new domain.

Transfer learning describes the reuse relationship, while fine tuning is one way to adapt state with new data or objectives. The adapted result is a new model instance even if people continue to use the base model's family name. Its behavior depends on the adaptation data, method, stopping, and surrounding pipeline, and it needs separate evaluation.

Foundation models are models trained on broad data at scale and adapted to many downstream tasks. The Stanford report that popularized the term emphasizes both their broad reuse and the way defects in a foundation model can propagate into many adapted systems.^[16] A foundation model is still not a complete application. Prompts, retrieval, tools, adapters, filters, and user interaction can dominate downstream behavior.

Compression and specialization create further identity questions. Quantization changes numerical representation and sometimes operations. Knowledge distillation trains a separate student to reproduce selected behavior. Pruning, conversion, and compilation can also produce derived artifacts. A derivative may preserve performance within measured tolerances, but it is not automatically the same model for audit, licensing, or safety purposes. Each artifact should record its parent, transformation, exact state, runtime requirements, and validation results.

Access to weights is only one layer of access. Reproducing or governing a model may also require architecture code, tokenizer, preprocessing, training method, data information, evaluation assets, and license terms. "Open weights," "open source," reproducible, and freely usable are separate claims.

The practical naming rule is simple. Use "model" for the specified representation or fitted object that performs inference. Use "architecture" or "family" for the shared structural design, "weights" or "state" for stored learned values, "checkpoint" for a training snapshot, "package" for a distributable execution bundle, and "system" or "service" for the operational whole. When a boundary differs, state it explicitly.

References

1. ^OECD. "Explanatory Memorandum on the Updated OECD Definition of an AI System." OECD Artificial Intelligence Papers No. 8, 2024. oecd.org/...623da898-en.pdf
2. ^Goodfellow, I., Y. Bengio, and A. Courville. "Optimization for Training Deep Models." In Deep Learning, Chapter 8, MIT Press, 2016. deeplearningbook.org/...optimization
3. ^Bergstra, J., and Y. Bengio. "Random Search for Hyper-Parameter Optimization." Journal of Machine Learning Research 13, 2012. jmlr.org/...bergstra12a.pdf
4. ^Breiman, L. "Statistical Modeling: The Two Cultures." Statistical Science 16(3), 2001. projecteuclid.org/...1009213726.pdf
5. ^Cawley, G. C., and N. L. C. Talbot. "On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation." Journal of Machine Learning Research 11, 2010. jmlr.org/...cawley10a
6. ^Guo, C., G. Pleiss, Y. Sun, and K. Q. Weinberger. "On Calibration of Modern Neural Networks." Proceedings of the 34th International Conference on Machine Learning, 2017. proceedings.mlr.press/...guo17a
7. ^D'Amour, A. et al. "Underspecification Presents Challenges for Credibility in Modern Machine Learning." Journal of Machine Learning Research 23, 2022. jmlr.org/...20-1335
8. ^Koh, P. W. et al. "WILDS: A Benchmark of in-the-Wild Distribution Shifts." Proceedings of the 38th International Conference on Machine Learning, 2021. proceedings.mlr.press/...koh21a
9. ^Mitchell, M. et al. "Model Cards for Model Reporting." Proceedings of the Conference on Fairness, Accountability, and Transparency, 2019. research.google/...model-cards-for-model-reporting
10. ^Sculley, D. et al. "Hidden Technical Debt in Machine Learning Systems." Advances in Neural Information Processing Systems 28, 2015. papers.nips.cc/...896fcaf2674f757a2463eba-Abstract
11. ^Pineau, J. et al. "Improving Reproducibility in Machine Learning Research." Journal of Machine Learning Research 22, 2021. jmlr.org/...20-303
12. ^ONNX. "Open Neural Network Exchange Intermediate Representation Specification." ONNX 1.23.0 documentation, accessed July 28, 2026. onnx.ai/...IR
13. ^PyTorch. "Saving and Loading Models." PyTorch 2.13 documentation, accessed July 28, 2026. docs.pytorch.org/...saving_loading_models
14. ^Tabassi, E. "Artificial Intelligence Risk Management Framework (AI RMF 1.0)." NIST AI 100-1, 2023. nvlpubs.nist.gov/...NIST.AI.100-1.pdf
15. ^Vassilev, A. et al. "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations." NIST AI 100-2e2025, corrected edition, 2025. doi.org/...NIST.AI.100-2e2025
16. ^Bommasani, R. et al. "On the Opportunities and Risks of Foundation Models." Stanford Center for Research on Foundation Models, 2021. crfm.stanford.edu/report
17. ^Yosinski, J., J. Clune, Y. Bengio, and H. Lipson. "How Transferable Are Features in Deep Neural Networks?" Advances in Neural Information Processing Systems 27, 2014. proceedings.neurips.cc/...c93f8580abbb330-Abstract