Computer Vision

Computer vision is the study of computational methods that extract, estimate, or generate useful representations from visual measurements. Inputs can include photographs, video, depth maps, thermal images, medical scans, or synchronized data from several sensors. Outputs range from a label for an entire image to object locations, pixel masks, motion fields, three-dimensional geometry, uncertainty estimates, or control-relevant state. The field overlaps Artificial Intelligence, Machine Learning, Pattern Recognition, and Robotics, but it also includes geometric and signal-based methods that do not require learned models.^[1]

A camera records only a projection of a scene. Illumination, viewpoint, occlusion, sensor response, and scene geometry can produce similar pixel values, so recovering the underlying world is generally an underdetermined inverse problem. A useful vision system therefore depends on assumptions about image formation, data, task boundaries, and the conditions under which its output will be used. A high score on one benchmark does not establish general visual understanding or safe operation in a different environment.

Modern systems often learn representations from large datasets, then adapt them to a particular task. Convolutional networks, vision transformers, self-supervised learning, and image-text pretraining are major approaches, but classical features, multiview geometry, optimization, and probabilistic estimation remain important. The choice among them depends on the output required, available supervision, latency and memory limits, failure costs, and whether the deployment distribution matches the evaluation data.

Scope and problem formulation

Computer vision begins with a measurement model. In a conventional camera, three-dimensional scene points are projected onto a two-dimensional sensor, while surface reflectance, lighting, optics, exposure, sampling, and noise determine recorded intensity. Other sensors measure different quantities: stereo cameras provide paired projections, lidar samples range, event cameras report brightness changes, and medical scanners reconstruct signals acquired through modality-specific physics. Calibration estimates parameters needed to relate measurements to rays, coordinates, or physical units.^[1]

The same image can support different questions. Image Recognition may ask which known concepts are present. Geometric vision may ask where the camera is, how points move, or what three-dimensional structure could have produced the views. A complete system can combine perception with tracking, planning, databases, language, or human review. A Multimodal Model may align images with text or other modalities, but multimodal reasoning is broader than visual perception alone.

Three distinctions prevent common category errors:

• Image processing and computer vision: Image processing transforms an image, for example by denoising or correcting color. Computer vision usually estimates information about a scene, object, event, or document. A processing operation can be a component of a vision pipeline.
• Recognition and geometry: Recognition predicts semantic categories or relationships. Geometry estimates quantities such as camera pose, depth, motion, or shape. Many systems combine both.
• Model and system: A trained model is one component. Sensors, preprocessing, thresholds, temporal logic, user interfaces, fallback behavior, and downstream decisions can change system performance and risk.

The output should match the decision. A classifier that assigns one label to a crop cannot by itself locate every object in a street scene. A detector that returns boxes does not specify exact boundaries. A monocular depth model estimates depth from learned regularities but cannot remove every scale ambiguity. Claims about performance are meaningful only when the input distribution, label space, output definition, metric, threshold, and evaluation protocol are specified.

Historical development

Early computer vision emphasized explicit representations and geometric reasoning. David Marr's 1982 framework separated the computational goal, the algorithm and representation, and the physical implementation. It described a progression from image-based descriptions toward surfaces and three-dimensional objects.^[2] This framework did not prescribe today's learning systems, but its insistence on defining what is computed remains relevant.

Classical methods constructed features by design. Canny formulated edge detection using detection, localization, and single-response criteria, exposing a tradeoff between accurate detection and localization.^[3] SIFT detected scale-space keypoints and described local neighborhoods so that corresponding structures could be matched across changes in scale and orientation.^[4] Histograms of oriented gradients summarized locally normalized gradient directions and were evaluated as features for pedestrian detection.^[5] Such features were paired with nearest-neighbor matching, robust geometric estimation, support vector machines, graphical models, and multiscale search. They are still useful when interpretability, limited data, or geometric correspondence matters.

Convolutional networks brought feature learning into end-to-end recognition. LeCun and colleagues described convolutional architectures and gradient-based training for document recognition in 1998.^[6] The 2009 ImageNet paper introduced a large, WordNet-organized image database and a crowd-based annotation process.^[7] In the 2012 ImageNet competition, AlexNet used a deep convolutional network trained with graphics processors and achieved a substantially lower top-5 error than the other submitted systems under that competition's protocol.^[8][9] This result accelerated adoption of Deep Learning, but it was a result on a specific closed-set classification benchmark, not evidence that vision had been solved.

Later architectures addressed optimization and task structure. ResNet reformulated stacked layers as residual functions with shortcut connections, enabling the authors to train substantially deeper networks and transfer learned features to detection.^[10] Faster R-CNN integrated learned region proposals with a detector.^[11] Fully convolutional networks adapted classification networks to dense pixel prediction, while Mask R-CNN added a mask branch to a region-based detector.^[12][13]

Transformers expanded the design space. The Vision Transformer represented an image as a sequence of patches and applied a Transformer encoder; its reported results depended on large-scale pretraining and transfer.^[14] DETR formulated detection as set prediction with bipartite matching and a Transformer encoder-decoder, reducing reliance on anchors and non-maximum suppression in that design.^[15] These developments did not make convolutions obsolete. Architecture comparisons depend on model scale, training data, augmentation, optimization, resolution, hardware, and the downstream task.

Learning paradigms and architectures

A Convolutional Neural Network applies shared filters across spatial locations. Weight sharing gives a useful translation-related inductive bias and reduces the number of parameters relative to an unstructured dense mapping. Pooling, stride, dilation, feature pyramids, and skip connections determine how resolution and receptive field change. Convolution does not by itself guarantee invariance to viewpoint, rotation, deformation, or illumination.

Vision transformers turn patches or learned visual tokens into sequences and use attention to mix information across locations. Hierarchical and windowed variants add multiscale structure or restrict attention for efficiency. In practice, convolutional and attention-based components are often combined. The architecture name alone is not enough to infer accuracy, memory use, latency, robustness, or suitability for a device.

Learning regimes differ in where supervision comes from:

• Supervised learning uses task labels such as classes, boxes, masks, or keypoints. It can directly optimize the desired output, but annotation policies and label errors become part of the learned problem.
• Self-Supervised Learning derives training signals from the data. SimCLR learned image representations by contrasting augmented views and showed that augmentation composition was central to its results.^[16] DINO used self-distillation without labels and reported semantic structure in attention maps from self-supervised vision transformers.^[17] Masked autoencoders learned by reconstructing hidden image patches with an asymmetric encoder-decoder.^[18]
• Transfer learning initializes a task from representations learned elsewhere. It can reduce task-specific labeling needs, but Transfer Learning can also transfer irrelevant shortcuts or biases when source and target conditions differ.
• Multimodal pretraining aligns images with language or other signals. CLIP learned from image-text pairs and used text prompts to define zero-shot classification tasks.^[19] Its results varied by dataset and prompt, and its authors separately audited training-evaluation overlap rather than assuming that web-scale data was clean.
• Promptable prediction conditions a model on points, boxes, masks, text, or examples. Segment Anything combined a promptable segmentation model with a data collection process and evaluated zero-shot transfer across segmentation settings.^[20] Promptability changes the interface, not the need to validate outputs in the target domain.

Training usually minimizes an empirical loss over sampled data. Cross-entropy is common for classification; detection combines classification with box or set losses; segmentation uses pixelwise, overlap, or region losses; geometry may minimize photometric, correspondence, or reprojection errors. Data Augmentation encodes assumptions by transforming training samples. If an augmentation changes a label's meaning, it can harm learning. If it fails to represent deployment variation, it cannot establish robustness to that variation.

Tasks and outputs

Computer vision is a collection of tasks rather than one universal prediction problem.

Scroll sideways for more →

Object and scene tasks evolved with benchmark definitions. PASCAL VOC provided shared datasets and evaluation procedures for classification and detection.^[21] COCO Dataset emphasized common objects in complex scenes and supplied instance segmentations as well as boxes.^[22] Panoptic segmentation later unified semantic labels for all pixels with instance identities for countable objects, together with the panoptic quality metric.^[25] Dedicated articles should be consulted for detector or segmenter lineages; the field-level distinction is the output and evaluation protocol.

Data, benchmarks, and metrics

Datasets determine what a model can observe and what a benchmark rewards. ImageNet organized images by WordNet synsets and used human annotation at a scale that enabled large classification experiments.^[7] PASCAL VOC standardized a smaller object-recognition challenge.^[21] COCO placed objects in natural context and annotated individual instances.^[22] Cityscapes collected stereo street sequences from 50 cities and provided fine and coarse pixel annotations for urban scene understanding.^[23] Kinetics increased the scale of human-action video classification and supported experiments with spatiotemporal networks.^[24] These datasets differ in geography, time period, capture process, licenses, label ontology, and intended task, so a score from one cannot be substituted for another.

Common metrics answer different questions:

• Accuracy counts correct decisions at a specified rule. Top-k accuracy accepts a prediction when the reference label appears among the k highest scores. It can hide class imbalance and does not measure probability quality.
• Precision is the share of positive predictions that are correct, while Recall is the share of reference positives found. The F1 Score is their harmonic mean at a chosen threshold. All three depend on the positive class and decision threshold.
• Average precision summarizes a precision-recall curve. Detection AP also depends on how predictions are matched to references and which intersection-over-union thresholds, object sizes, and averaging rules the benchmark uses.^[21][22]
• Intersection over Union measures overlap divided by union for regions or masks. Mean IoU averages over classes according to a stated convention. Dice gives related overlap weighting but is not numerically identical.
• Panoptic quality combines recognition and segmentation components for matched segments under the protocol introduced with the task.^[25]
• Calibration asks whether confidence values correspond to empirical correctness frequencies. A model can improve accuracy while remaining poorly calibrated, and a calibrator fitted in one distribution can degrade after shift.^[42][43]

Test data should be isolated from model selection and training. Repeated use of a public test set can turn it into an implicit development target. Recht and colleagues recreated CIFAR-10 and ImageNet test sets using the original collection procedures and observed accuracy drops while preserving a relationship between old and new performance.^[40] Northcutt and colleagues found human-validated label errors across ten widely used test sets and showed that corrected labels could affect model rankings.^[41] With web-scale pretraining, exact or near-duplicate benchmark images can also enter training corpora; the CLIP paper included an overlap analysis to estimate this effect.^[19]

Evaluation leakage includes duplicate scenes split across train and test, frames from the same video on both sides of a split, multiple images from one patient across medical partitions, tuning thresholds on the test set, or selecting a model after repeated test feedback. Grouped splits by person, site, device, video, location, or time are often more informative than random image splits. A benchmark report should disclose the dataset version, split construction, preprocessing, label policy, metric implementation, confidence intervals or variability, and every source of training data.

Three-dimensional and temporal vision

Geometric vision estimates structure and motion from relationships among views. Camera calibration defines the mapping between image coordinates and rays. Stereo uses corresponding points in paired views to infer disparity and depth. Structure from motion jointly estimates camera poses and a sparse or dense scene from overlapping images. The COLMAP reconstruction system revisited structure-from-motion design with attention to correspondence verification, triangulation, and incremental reconstruction.^[26] Simultaneous Localization and Mapping combines mapping with estimating a moving sensor's pose, often fusing cameras with inertial or range sensors.

Learned scene representations complement explicit geometry. NeRF represented a scene as a continuous function of location and view direction, then used differentiable volume rendering to synthesize views from images with known camera poses.^[27] Such a representation can produce high-quality novel views under its experimental assumptions, but view synthesis quality is not the same as metrically accurate geometry. Pose quality, scene dynamics, lighting changes, training-view coverage, and render cost all affect results.

Video adds time and association. A system may classify a short clip, localize an action in an untrimmed stream, track objects, estimate camera motion, segment moving instances, or forecast future state. Kinetics was designed for clip-level human action classification, while its I3D experiments inflated two-dimensional filters into spatiotemporal networks.^[24] Dense optical flow estimates apparent image motion rather than physical object velocity. RAFT built all-pairs feature correlations and iteratively updated a flow field, illustrating how learned matching and iterative refinement can be combined.^[28]

Temporal evaluation must prevent near-duplicate leakage and identity leakage across splits. Frame accuracy can overstate performance when neighboring frames are highly correlated. Tracking metrics may trade detection quality against identity continuity. Long-horizon systems also need tests for reappearance, occlusion, camera cuts, latency, and accumulated error.

Applications and deployment

Computer vision is used for document conversion, manufacturing inspection, remote sensing, accessibility, scientific imaging, media indexing, sports analysis, robotics, and medical image analysis. Each application supplies a different ground truth and cost of error. A missed defect, an incorrect crop suggestion, and a false clinical alert are not equivalent even when they use similar classifiers.

In medical imaging, retrospective studies can demonstrate a narrowly defined capability without establishing clinical benefit. Esteva and colleagues compared a skin-lesion classifier with 21 dermatologists on two binary image-classification tasks using biopsy-proven images.^[29] The result was specific to those datasets, tasks, readers, and operating points. Zech and colleagues later showed that pneumonia classifiers could identify hospital-specific signals and perform worse on external hospital data than on internal tests.^[30] These findings support patient-separated and site-separated validation, assessment across equipment and subgroups, prospective workflow studies, and monitoring after deployment. International medical-device guidance treats machine-learning quality as a total-product-lifecycle concern, not a one-time benchmark score.^[31]

For an Autonomous Vehicle or mobile robot, perception is one subsystem among sensing, localization, prediction, planning, control, and safety mechanisms. A detector score alone does not measure end-to-end collision risk. Relevant tests include weather, lighting, geography, rare objects, sensor faults, latency, temporal consistency, uncertainty handling, and the behavior of downstream components when perception abstains or fails.

Industrial inspection requires representative defects and attention to changes in materials, camera placement, production lines, and defect prevalence. Remote sensing requires geospatial and temporal separation because nearby tiles or repeated observations can leak information. Document systems must evaluate layout, language, handwriting, scan quality, and structured extraction rather than character recognition alone. In every domain, the target population and decision process should determine the test set, threshold, and human-review design.

Robustness, bias, privacy, and security

In-distribution accuracy does not characterize all failures. Distribution Shift occurs when deployment inputs, labels, or relationships differ from development data. ImageNet-C and ImageNet-P were created to test common corruptions and perturbations separately from worst-case adversarial attacks.^[32] Shortcut learning describes decision rules that score well on a benchmark by exploiting correlations that do not transfer to the intended environment.^[33] Background, watermark, acquisition device, hospital marker, or annotation artifact can all become predictive without representing the concept a developer intended.

Adversarial research shows a different failure mode. Szegedy and colleagues demonstrated that small optimized perturbations could cause neural-network misclassification and could transfer between models in their experiments.^[34] Physical studies subsequently evaluated altered road signs under selected viewpoints and capture conditions.^[35] An Adversarial Attack is defined by an attacker's knowledge, access, objective, and allowed perturbation. Robustness to one threat model does not imply robustness to another or to ordinary corruption.

The model supply chain is also part of security. BadNets showed that malicious training could implant trigger-dependent behavior while ordinary validation performance remained high.^[36] Mitigations therefore include provenance for data and weights, access controls, integrity checks, isolated evaluation, review of dependencies and preprocessing, red-team tests tied to credible threats, and monitoring for abnormal inputs. No single detector or benchmark proves that a model is free of backdoors.

Bias requires task-specific measurement. Gender Shades reported large intersectional error disparities for three commercial gender classifiers on its constructed benchmark.^[37] NIST's 2019 face-recognition evaluation found demographic differentials for many, but not all, submitted algorithms and emphasized distinctions among one-to-one verification and one-to-many identification.^[38] These results do not justify one universal claim about every visual model. They demonstrate why Algorithmic Bias should be evaluated by relevant subgroups, error types, thresholds, image quality, and use context. For Facial Recognition, false matches and false nonmatches can have different consequences depending on whether a human reviews candidates and what action follows.

Visual data can contain faces, homes, locations, health information, documents, and bystanders who did not expect model training or secondary analysis. Privacy risks arise during collection, annotation, sharing, model access, and retention. Membership-inference research demonstrated that an attacker can sometimes infer whether a record was in a model's training data, with success depending on the model, data, and access setting.^[39] Privacy assessment should therefore cover lawful and consent-compatible collection, minimization, retention, access, de-identification limits, memorization tests, output controls, and the possibility that visual context re-identifies a person even after direct identifiers are removed.

Reproducibility and responsible evaluation

A reproducible computer-vision result identifies what was trained, on which data, with which code path, and how the reported number was selected. At minimum, a report should include dataset versions and licenses, split logic, preprocessing and augmentation, architecture and initialization, loss functions, optimizer and schedule, batch size, image resolution, stopping and checkpoint-selection rules, random seeds, hardware and software versions, inference settings, post-processing, thresholds, and metric implementation. Compute and latency claims should include device, precision, batch size, input size, warmup, preprocessing, and whether transfers or post-processing are timed.

Datasets should document motivation, composition, collection, annotation, recommended uses, known gaps, maintenance, and access conditions, following the rationale of datasheets for datasets.^[44] A released model should state intended uses, out-of-scope uses, training and evaluation data, subgroup and condition-specific results, limitations, and ethical or safety considerations. A Model Card is one structure for this disclosure.^[45]

Reproducibility does not mean that one run must produce identical floating-point values on every platform. It means that evidence is detailed enough to repeat the procedure, diagnose differences, and determine whether conclusions survive reasonable variation. The NeurIPS reproducibility program combined code submission, an independent reproduction challenge, and a checklist intended to improve reporting and verification.^[46] For empirical vision work, multiple runs or uncertainty intervals are especially important when gains are small relative to training variation.

Before deployment, evaluation should move beyond the development benchmark:

1. Define the decision, users, failure costs, and abstention or fallback path.
2. Construct splits that isolate people, devices, scenes, sites, or time periods as the use case requires.
3. Test subgroup performance, calibration, common corruptions, distribution shifts, and credible attacks.
4. Evaluate the complete system, including sensors, preprocessing, post-processing, interfaces, and downstream actions.
5. Record model and data versions, monitor changes in inputs and outcomes, and specify conditions that trigger review or rollback.

Computer vision performance is therefore conditional evidence, not a permanent property of an architecture. Reliable claims connect a defined task to representative data, an auditable protocol, and the environment in which the output will be used.

References

1. ^Richard Szeliski. Computer Vision: Algorithms and Applications. 2nd edition, 2022. szeliski.org/Book
2. ^David Marr. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. MIT Press, 1982. mitpress.mit.edu/...vision
3. ^John Canny. "A Computational Approach to Edge Detection." IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986. doi.org/...TPAMI.1986.4767851
4. ^David G. Lowe. "Distinctive Image Features from Scale-Invariant Keypoints." International Journal of Computer Vision, 2004. doi.org/...B:VISI.0000029664.99615.94
5. ^Navneet Dalal and Bill Triggs. "Histograms of Oriented Gradients for Human Detection." CVPR, 2005. doi.org/...CVPR.2005.177
6. ^Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. "Gradient-Based Learning Applied to Document Recognition." Proceedings of the IEEE, 1998. doi.org/...5.726791
7. ^Jia Deng et al. "ImageNet: A Large-Scale Hierarchical Image Database." CVPR, 2009. image-net.org/...imagenet_cvpr09.pdf
8. ^Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. "ImageNet Classification with Deep Convolutional Neural Networks." NeurIPS, 2012. proceedings.neurips.cc/...8436e924a68c45b-Abstract
9. ^Olga Russakovsky et al. "ImageNet Large Scale Visual Recognition Challenge." International Journal of Computer Vision, 2015. doi.org/...s11263-015-0816-y
10. ^Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Deep Residual Learning for Image Recognition." CVPR, 2016. openaccess.thecvf.com/..._Learning_CVPR_2016_paper
11. ^Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks." NeurIPS, 2015. proceedings.neurips.cc/...ba028a21ed38046-Abstract
12. ^Jonathan Long, Evan Shelhamer, and Trevor Darrell. "Fully Convolutional Networks for Semantic Segmentation." CVPR, 2015. openaccess.thecvf.com/..._Networks_2015_CVPR_paper
13. ^Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick. "Mask R-CNN." ICCV, 2017. openaccess.thecvf.com/...ask_R-CNN_ICCV_2017_paper
14. ^Alexey Dosovitskiy et al. "An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale." ICLR, 2021. openreview.net/forum
15. ^Nicolas Carion et al. "End-to-End Object Detection with Transformers." ECCV, 2020. arxiv.org/...2005.12872
16. ^Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. "A Simple Framework for Contrastive Learning of Visual Representations." ICML, 2020. proceedings.mlr.press/...chen20j
17. ^Mathilde Caron et al. "Emerging Properties in Self-Supervised Vision Transformers." ICCV, 2021. openaccess.thecvf.com/...nsformers_ICCV_2021_paper
18. ^Kaiming He et al. "Masked Autoencoders Are Scalable Vision Learners." CVPR, 2022. openaccess.thecvf.com/..._Learners_CVPR_2022_paper
19. ^Alec Radford et al. "Learning Transferable Visual Models From Natural Language Supervision." ICML, 2021. proceedings.mlr.press/...radford21a
20. ^Alexander Kirillov et al. "Segment Anything." ICCV, 2023. openaccess.thecvf.com/..._Anything_ICCV_2023_paper
21. ^Mark Everingham et al. "The PASCAL Visual Object Classes Challenge." International Journal of Computer Vision, 2010. doi.org/...s11263-009-0275-4
22. ^Tsung-Yi Lin et al. "Microsoft COCO: Common Objects in Context." ECCV, 2014. arxiv.org/...1405.0312
23. ^Marius Cordts et al. "The Cityscapes Dataset for Semantic Urban Scene Understanding." CVPR, 2016. openaccess.thecvf.com/...s_Dataset_CVPR_2016_paper
24. ^Joao Carreira and Andrew Zisserman. "Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset." CVPR, 2017. openaccess.thecvf.com/...is_Action_CVPR_2017_paper
25. ^Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollar. "Panoptic Segmentation." CVPR, 2019. openaccess.thecvf.com/...mentation_CVPR_2019_paper
26. ^Johannes L. Schonberger and Jan-Michael Frahm. "Structure-from-Motion Revisited." CVPR, 2016. openaccess.thecvf.com/...Revisited_CVPR_2016_paper
27. ^Ben Mildenhall et al. "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis." ECCV, 2020. arxiv.org/...2003.08934
28. ^Zachary Teed and Jia Deng. "RAFT: Recurrent All-Pairs Field Transforms for Optical Flow." ECCV, 2020. arxiv.org/...2003.12039
29. ^Andre Esteva et al. "Dermatologist-Level Classification of Skin Cancer with Deep Neural Networks." Nature, 2017. doi.org/...nature21056
30. ^John R. Zech et al. "Variable Generalization Performance of a Deep Learning Model to Detect Pneumonia in Chest Radiographs: A Cross-Sectional Study." PLOS Medicine, 2018. doi.org/...journal.pmed.1002683
31. ^International Medical Device Regulators Forum. "Good Machine Learning Practice for Medical Device Development: Guiding Principles." 2025. imdrf.org/...device-development-guiding-principles
32. ^Dan Hendrycks and Thomas Dietterich. "Benchmarking Neural Network Robustness to Common Corruptions and Perturbations." ICLR, 2019. arxiv.org/...1903.12261
33. ^Robert Geirhos et al. "Shortcut Learning in Deep Neural Networks." Nature Machine Intelligence, 2020. doi.org/...s42256-020-00257-z
34. ^Christian Szegedy et al. "Intriguing Properties of Neural Networks." ICLR, 2014. arxiv.org/...1312.6199
35. ^Kevin Eykholt et al. "Robust Physical-World Attacks on Deep Learning Visual Classification." CVPR, 2018. openaccess.thecvf.com/...d_Attacks_CVPR_2018_paper
36. ^Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. "BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain." 2017. arxiv.org/...1708.06733
37. ^Joy Buolamwini and Timnit Gebru. "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification." FAT, 2018. proceedings.mlr.press/...buolamwini18a
38. ^Patrick Grother, Mei Ngan, and Kayee Hanaoka. "Face Recognition Vendor Test Part 3: Demographic Effects." NISTIR 8280, 2019. doi.org/...NIST.IR.8280
39. ^Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. "Membership Inference Attacks Against Machine Learning Models." IEEE Symposium on Security and Privacy, 2017. doi.org/...SP.2017.41
40. ^Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. "Do ImageNet Classifiers Generalize to ImageNet?" ICML, 2019. proceedings.mlr.press/...recht19a
41. ^Curtis G. Northcutt, Anish Athalye, and Jonas Mueller. "Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks." NeurIPS Datasets and Benchmarks, 2021. arxiv.org/...2103.14749
42. ^Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. "On Calibration of Modern Neural Networks." ICML, 2017. proceedings.mlr.press/...guo17a
43. ^Yaniv Ovadia et al. "Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift." NeurIPS, 2019. proceedings.neurips.cc/...371888657d2eb1d-Abstract
44. ^Timnit Gebru et al. "Datasheets for Datasets." Communications of the ACM, 2021. arxiv.org/...1803.09010
45. ^Margaret Mitchell et al. "Model Cards for Model Reporting." FAT, 2019. arxiv.org/...1810.03993
46. ^Joelle Pineau et al. "Improving Reproducibility in Machine Learning Research: A Report from the NeurIPS 2019 Reproducibility Program." Journal of Machine Learning Research, 2021. jmlr.org/...20-303