Coverage Bias
Last reviewed
Sources
22 citations
Review status
Source-backed
Revision
v3 ยท 4,985 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Sources
22 citations
Review status
Source-backed
Revision
v3 ยท 4,985 words
Add missing citations, update stale details, or suggest a clearer explanation.
Coverage bias is a type of selection bias that occurs when the method used to collect data systematically excludes part of the target population, so the sample does not "cover" everyone the conclusions are meant to describe. Google's Machine Learning Crash Course defines it concisely: "Coverage bias occurs if data is not selected in a representative fashion."[1] It arises when certain segments of a population are absent from, or underrepresented in, the dataset used for analysis or model training, typically because the sampling frame (the list or mechanism the sample is drawn from) does not match the intended target population. The term originates in survey methodology but has become equally central to artificial intelligence, where training data scraped from non-representative sources produces models that perform poorly or unfairly for the missing groups. The classic illustration is the 1936 Literary Digest poll, which surveyed 10 million people drawn from telephone, automobile, and club lists and predicted the wrong winner of the U.S. presidential election by nearly 20 percentage points because those lists excluded poorer voters.[2]
Imagine you want to find out what every kid in your school likes to eat for lunch. But instead of asking everyone, you only ask the kids who sit at your table. The kids at your table might all love pizza, so you would think the whole school loves pizza. But the kids at other tables might prefer tacos, sushi, or sandwiches. Because you only asked some kids and missed all the others, your answer is wrong. That is coverage bias: when you leave out a big chunk of the people you are trying to learn about, and the information you collect does not match what the whole group actually thinks or does.
In survey methodology, coverage bias falls within the Total Survey Error (TSE) framework, originally articulated by Robert Groves in his 1989 book Survey Errors and Survey Costs.[3] The TSE framework classifies errors into two broad categories: errors of nonobservation (which include coverage error, sampling error, and nonresponse error) and errors of observation (which include measurement error).
Coverage error is defined as the discrepancy between the target population (the group about which conclusions are desired) and the sampling frame (the list or mechanism from which the sample is actually drawn). When the sampling frame fails to include all members of the target population, the result is undercoverage. When the frame includes members who do not belong to the target population, the result is overcoverage.
Formally, coverage bias for a population mean can be expressed as:
$$B_{coverage} = \bar{Y}{frame} - \bar{Y}{target}$$
where $\bar{Y}{frame}$ is the mean of the variable of interest across units included in the sampling frame, and $\bar{Y}{target}$ is the true population mean. This bias is nonzero only when the excluded (or over-included) units differ systematically from the included units on the variable being measured.
Coverage bias is one of several types of bias that arise during data collection. Google's Machine Learning Crash Course groups coverage bias, sampling bias, and non-response bias together as the three components of selection bias, all of which leave the data unrepresentative of the real-world population.[1] Coverage bias is distinct from each: it happens before sampling begins, when groups are excluded from the frame entirely. The table below summarizes the differences.
| Bias type | Definition | When it occurs | Example |
|---|---|---|---|
| Coverage bias | Mismatch between the target population and the sampling frame | Before sampling begins; certain groups are excluded from the frame entirely | A telephone survey that cannot reach households without phones |
| Sampling bias | Non-random selection of units from within the sampling frame | During the sampling process; some frame members have unequal selection probabilities | Surveying only the first 200 respondents to reply to an email[1] |
| Nonresponse bias | Systematic differences between respondents and non-respondents | After sampling; selected units refuse or fail to participate | Political poll where supporters of one candidate are less likely to respond |
| Reporting bias | Distorted frequency of events or outcomes in the collected data | During data recording; certain outcomes are more likely to be documented | Medical literature that overreports positive drug trial results |
| Measurement bias | Systematic errors in how data values are recorded or observed | During data collection; instruments or methods introduce consistent errors | A faulty sensor that consistently reads 2 degrees too high |
| Historical bias | Training data reflects past inequities or outdated social norms | Inherent in the data; historical patterns encode discrimination | Loan approval models trained on decades of discriminatory lending decisions |
Coverage bias can enter a dataset through multiple pathways. Understanding these causes is essential for identifying and addressing the problem.
The most direct cause of coverage bias is a sampling frame that fails to include all members of the target population. In traditional survey research, common frames include telephone directories, voter registration lists, and mailing address databases. Each of these frames excludes certain population segments. For example, telephone directories miss people with unlisted numbers and those who rely exclusively on mobile phones. Voter registration lists exclude eligible citizens who have not registered. Building a complete frame is expensive even for national governments: the U.S. Census Bureau's 2010 address canvassing operation cost about $444 million and employed roughly 140,000 temporary field workers to verify more than 150 million addresses, yet still required follow-up operations to catch addresses the initial canvass missed.[4]
The method used to collect data can systematically favor certain groups. Online surveys, for instance, can only reach individuals with internet access. According to the International Telecommunication Union, about 5.4 billion people (67% of the world's population) were online in 2023, which still left an estimated 2.6 billion people offline, disproportionately concentrated in low-income countries and rural areas. The divide is stark: 93% of people in high-income countries used the internet in 2023, compared with just 27% in low-income countries.[5] Similarly, data scraped from social media platforms overrepresents younger, more digitally active demographics and underrepresents older adults and people in regions where those platforms are less popular.
Datasets collected from specific geographic regions may not generalize to other areas. This is a significant concern in autonomous driving, where most training data has been collected in the United States, Western Europe, and East Asia. A well-known example involves Volvo's experimental self-driving cars: when the company tried to extend its animal-detection system (developed for slower Swedish wildlife such as moose and reindeer) to Australia in 2015, engineers found that the kangaroo's hopping motion confused the system, because a kangaroo in mid-air appears farther away than when it lands, and that motion pattern was absent from the original training data.[6]
Populations change over time, and data collected at one point may not represent the population at a later point. Consumer preferences shift, demographics evolve, and new subgroups emerge. A model trained on data from five years ago may exhibit coverage bias with respect to the current population simply because the world has changed. Longitudinal studies are particularly vulnerable to this problem because participants relocate, become unreachable, or drop out, creating a survivorship effect in the remaining sample.
When data is collected through a particular platform or technology, the characteristics of that platform's user base shape the resulting dataset. Twitter (now X) data, for example, has been widely used in sentiment analysis and public opinion research, but Twitter users skew younger, more urban, more educated, and more politically engaged than the general population. Any model trained on Twitter data inherits these biases.
In natural language processing, training corpora are heavily skewed toward high-resource languages, especially English. Although roughly 7,000 languages are spoken worldwide, only about 20 of them are well-resourced enough to have large text corpora, and only around 100 have any meaningful NLP resources at all.[7] Low-resource languages spoken by smaller or marginalized communities are severely underrepresented. This creates models that work well for English speakers but perform poorly or not at all for speakers of languages like Khasi, Kashmiri, Mizo, or hundreds of others.
Coverage error manifests in three distinct forms, each with different implications for the resulting bias.
| Type | Description | Example | Effect on analysis |
|---|---|---|---|
| Undercoverage | Members of the target population are absent from the sampling frame | Homeless individuals excluded from address-based survey frames | Systematically underestimates characteristics of excluded groups |
| Overcoverage | Some members of the target population appear in the sampling frame more than once | A person with multiple phone numbers has a higher probability of selection in a phone survey | Overweights the characteristics of duplicated individuals |
| Inclusion of non-target units | The sampling frame contains members who do not belong to the target population | A voter survey frame that includes non-citizens who cannot vote | Dilutes the measured characteristics with irrelevant data |
Of these three types, undercoverage is generally considered the most serious because it can be difficult to detect and even more difficult to correct. If entire segments of the population are absent from the frame, no statistical technique applied to the collected data can fully recover information about those missing groups.
Coverage bias is one of the most consequential sources of error in machine learning, because models can only learn patterns present in their training data. It is a core form of bias in AI: when entire population segments are absent from training data, the resulting models may exhibit several failure modes.
A model trained on data that does not represent the full target population will typically perform well on data similar to its training set but fail when encountering underrepresented or absent groups. This gap between training performance and real-world performance is a direct consequence of coverage bias and is distinct from overfitting, which involves learning noise in the training data rather than missing entire subpopulations.
When some groups are underrepresented in training data, models tend to produce higher error rates for those groups. This has been documented in multiple domains.
| Domain | Finding | Source |
|---|---|---|
| Facial recognition | Commercial systems had error rates up to 34.7% for darker-skinned women compared to 0.8% for lighter-skinned men | Buolamwini & Gebru, 2018[8] |
| Speech recognition | Five major commercial systems averaged a 35% word error rate for Black speakers versus 19% for white speakers | Koenecke et al., 2020[9] |
| Chest X-ray analysis | Computer vision algorithms for diagnosing pathologies from chest X-rays performed worse for Black patients | Seyyed-Kalantari et al., 2021[10] |
| NLP sentiment analysis | Models trained primarily on formal English text struggled with African American Vernacular English and other dialects | Blodgett et al., 2016[11] |
| Dermatology AI | Skin lesion classification systems showed lower accuracy for dark-skinned individuals | Multiple studies, 2019-2023 |
The speech recognition disparity documented by Koenecke and colleagues is especially telling: across systems from Amazon, Apple, Google, IBM, and Microsoft, the average word error rate was 0.35 for Black speakers compared with 0.19 for white speakers, and the gap persisted even on identical phrases, which the authors traced to acoustic models that "have not been trained on enough audio data from black speakers."[9]
Coverage bias can create self-reinforcing cycles. When a biased model is deployed and its outputs are used to make decisions, those decisions can affect what data is collected in the future, perpetuating or even amplifying the original bias. For example, a predictive policing algorithm trained on historical arrest data (which reflects coverage bias due to differential policing practices across neighborhoods) may direct more police resources to already over-policed areas, leading to more arrests in those areas, which in turn generates more training data from those same areas.
Coverage bias is a primary driver of unfair algorithmic outcomes. The COMPAS recidivism prediction tool, analyzed by ProPublica in 2016, illustrated how biased data can lead to discriminatory predictions. Their analysis of more than 10,000 criminal defendants in Broward County, Florida, found that Black defendants were 77% more likely to be flagged as higher risk for future violent crime than white defendants, even after controlling for prior offenses, age, and gender.[12] While the COMPAS case involves multiple types of bias, coverage bias in the training data, where certain populations were overrepresented or underrepresented relative to the actual defendant population, contributed to the disparate outcomes.
Several well-documented cases illustrate how coverage bias has led to incorrect conclusions or harmful outcomes.
Perhaps the most famous example of coverage bias in history is the 1936 U.S. presidential election poll conducted by The Literary Digest. The magazine sent out 10 million straw ballots to predict the outcome of the election between Franklin D. Roosevelt and Alf Landon, compiling its mailing lists from telephone directories, automobile registration records, and club membership rolls. Based on about 2.3 million returned ballots, the poll predicted that Landon would win with 57% of the vote.[2]
Roosevelt instead won in a landslide with 62% of the popular vote. The poll's error stemmed from severe coverage bias: in 1936, at the height of the Great Depression, telephones, automobiles, and club memberships were luxuries concentrated among wealthier Americans, who tended to favor the Republican candidate. The sampling frame systematically excluded lower-income voters who overwhelmingly supported Roosevelt. The poll also suffered from nonresponse bias, as Roosevelt supporters were less motivated to return their ballots.
Meanwhile, George Gallup correctly predicted Roosevelt's victory using a much smaller sample of approximately 50,000 randomly selected citizens.[2] This episode demonstrated that increasing sample size cannot compensate for a biased sampling frame, and it helped establish the foundations of modern scientific polling.
The Chicago Tribune's premature headline declaring Thomas Dewey the winner of the 1948 presidential election was partly a result of coverage bias in telephone surveys. At that time, telephone ownership was concentrated among more affluent, urban households, and the resulting polls systematically underrepresented rural and lower-income voters who favored Harry Truman.
The ImageNet dataset, one of the most influential benchmarks in deep learning, has been shown to contain significant demographic imbalances. Research has found that people annotated as dark-skinned, female, and over 40 years old were underrepresented across most person-related categories. A 2021 study by Steed and Caliskan demonstrated that state-of-the-art image models pretrained on ImageNet learned humanlike biases about race, gender, and other attributes: the models iGPT and SimCLRv2 associated images of white people with tools while associating images of Black people with weapons.[13]
A widely cited example of coverage bias in healthcare AI involved an algorithm used across several U.S. health systems to identify patients who would benefit from additional care management. A 2019 study published in Science by Obermeyer et al. found that the algorithm exhibited racial bias: it was less likely to refer Black patients for extra care compared to equally sick white patients. The root cause was that the algorithm used healthcare spending as a proxy for health needs. Because Black patients historically had less access to healthcare and therefore lower spending, the training data exhibited coverage bias that reflected systemic inequities in healthcare access rather than actual health status. The authors estimated that correcting the bias would raise the share of Black patients flagged for extra help from 17.7% to 46.5%.[14]
Computer vision datasets frequently exhibit coverage bias along demographic, geographic, and contextual dimensions. The Gender Shades study by Buolamwini and Gebru (2018) demonstrated that commercial facial analysis systems from Microsoft, IBM, and Face++ had error rates for darker-skinned women that were 10 to 40 times higher than for lighter-skinned men, with the worst case reaching 34.7% versus 0.8%.[8] This disparity was traced back to training datasets that were predominantly composed of lighter-skinned subjects.
Beyond demographics, computer vision datasets also exhibit geographic coverage bias. Objects, architecture, vegetation, road markings, and signage vary across regions, but many popular datasets are dominated by images from North America and Europe. A model trained on such data may fail to recognize everyday objects in African, South Asian, or Latin American contexts.
Coverage bias in NLP extends beyond language selection to the demographic composition of text sources. Many foundational NLP datasets were created from established news sources such as the Wall Street Journal and the Frankfurter Rundschau, which represent a narrow slice of written language produced predominantly by educated, middle-class, white, male authors.[15] Models trained on these corpora perform significantly worse for content produced by younger writers, ethnic minorities, and speakers of non-standard dialects.
The problem extends to large language models trained on web-scraped data. While the internet provides a vast text corpus, it is not a representative sample of human language use. Internet text overrepresents English-speaking, younger, more educated, and more affluent populations. Languages spoken primarily in oral traditions or in regions with limited internet infrastructure are almost entirely absent.
Autonomous driving systems present a clear case study in geographic coverage bias. The majority of training data for self-driving cars has been collected in California, Western Europe, and parts of East Asia. This concentration means that autonomous systems may be optimized for the road conditions, traffic patterns, weather, signage, and pedestrian behavior typical of those regions but may struggle in other environments.
Specific challenges include the diversity of vehicles and road users across countries (electric scooters in China differ substantially from those in Germany), varying road infrastructure quality, and region-specific hazards. The Volvo kangaroo incident mentioned earlier illustrates how geographic coverage bias can create entirely unanticipated failure modes.
Recent efforts to address this problem include datasets like NVIDIA's PhysicalAI-Autonomous-Vehicles dataset, which contains 1,700 hours of driving data recorded across 25 countries and more than 2,500 cities.[16]
Clinical AI models frequently suffer from coverage bias because the populations represented in training data do not match the patient populations the models will serve. Clinical trial cohorts have historically skewed heavily white: one analysis of trials involving roughly 150,000 patients found the participant pool was about 86% white, even though medications can have different effects across genetic groups.[17] More than half of all published clinical AI models use datasets sourced from either the United States or China, limiting their applicability to patients in other regions.
Gender bias in clinical data is also a concern. The majority of clinical trial participants have historically been male, and gender-related biases during preclinical stages of drug development can alter how women respond to newly developed treatments. When AI models are trained on this skewed data, they may produce less accurate predictions for women.
Identifying coverage bias requires comparing the characteristics of the dataset against the known or estimated characteristics of the target population. Several approaches are commonly used.
The most straightforward detection method involves comparing the demographic composition of a dataset against census data or other population benchmarks. If a dataset intended to represent the general population contains 90% male subjects when the population is roughly 50% male, coverage bias is present. This approach requires access to reliable population statistics and clear definitions of relevant demographic categories.
Coverage bias often manifests as differential model performance across subgroups. By evaluating a model's accuracy, precision, recall, and other metrics separately for different demographic groups, analysts can identify groups for which the model underperforms. Significant performance disparities between groups are a strong signal of coverage bias in the training data.
Several quantitative metrics have been developed to measure fairness and detect bias in ML models.
| Metric | Definition | What it measures |
|---|---|---|
| Demographic parity | Equal positive prediction rates across groups | Whether the model's decisions are independent of group membership |
| Equal opportunity | Equal true positive rates across groups | Whether qualified members of each group have equal chances of positive outcomes |
| Predictive parity | Equal positive predictive values across groups | Whether positive predictions are equally reliable across groups |
| Error rate balance | Equal false positive and false negative rates across groups | Whether the model makes the same types of errors at equal rates for all groups |
| Disparate impact | Ratio of positive outcome rates between groups | Whether one group receives positive outcomes at a substantially lower rate |
Structured documentation practices can help identify potential coverage bias before models are trained. Datasheets for Datasets (Gebru et al., 2021) and Data Statements for NLP (Bender & Friedman, 2018) are frameworks that encourage dataset creators to document collection methods, population coverage, known limitations, and intended use cases.[18][19] By making these details explicit, downstream users can assess whether a dataset is appropriate for their application. The Datasheets framework poses more than 50 questions spanning a dataset's motivation, composition, collection process, preprocessing, uses, distribution, and maintenance.[18]
No single technique fully eliminates coverage bias, but a combination of approaches can substantially reduce its impact.
The most effective mitigation is to collect more representative data in the first place. This includes using multiple data sources (mixed-mode data collection), actively recruiting underrepresented groups, and designing collection procedures that minimize exclusion. In survey research, combining mail, telephone, and in-person interviews can reach populations that any single mode would miss.
Stratified sampling involves dividing the target population into homogeneous subgroups (strata) based on relevant characteristics and then sampling from each stratum in proportion to its share of the population. This ensures that all important subgroups are represented in the final dataset. It requires prior knowledge of the population composition and the relevant stratification variables.
When certain groups are known to be underrepresented, deliberate oversampling can compensate. The U.S. National Center for Health Statistics, for example, intentionally oversamples minority populations in its health surveys to ensure sufficient data for analysis of these groups. The Synthetic Minority Over-sampling Technique (SMOTE) and its variants provide algorithmic approaches to generating synthetic examples of underrepresented classes, though care must be taken to avoid introducing artificial patterns.
When the degree of underrepresentation can be quantified, sample weights can be applied to correct for coverage bias. Each observation is assigned a weight inversely proportional to its probability of appearing in the dataset. For example, if a dataset contains 20% women when the target population is 50% women, observations from women can be assigned higher weights to restore balance. This technique corrects for known biases but cannot address groups that are entirely absent from the data.
Data augmentation techniques can expand the diversity of a training dataset by creating modified versions of existing examples. In image recognition, this might involve rotating, cropping, adjusting lighting, or adding noise to existing images. While data augmentation can help reduce the impact of coverage bias, it cannot substitute for data from populations that are entirely unrepresented, since augmented data is derived from existing samples.
Domain adaptation techniques adjust models trained on one distribution (the source domain) to perform well on a different distribution (the target domain). Transfer learning, adversarial domain adaptation, and fine-tuning on target domain data are common approaches. These methods are particularly useful when collecting representative data from the target domain is expensive or impractical.
After a model has been trained, post-processing methods can adjust its outputs to satisfy fairness constraints. These methods modify prediction thresholds or calibrate outputs separately for different groups. While post-processing cannot fix the underlying data problem, it can reduce the downstream impact of coverage bias on decisions.
Bias mitigation methods are commonly classified by where they intervene in the machine learning pipeline.[20]
| Stage | Approach | Examples | Limitations |
|---|---|---|---|
| Pre-processing | Modify the training data before model training | Resampling, reweighting, data augmentation, stratified sampling | Cannot address entirely absent groups |
| In-processing | Modify the learning algorithm itself | Fairness constraints during optimization, adversarial debiasing, regularization | May reduce overall model accuracy |
| Post-processing | Modify model outputs after training | Threshold adjustment, output calibration, equalized odds post-processing | Does not fix underlying model or data issues |
Coverage bias has drawn increasing attention from regulators and policymakers. The European Union's AI Act (Regulation (EU) 2024/1689), which entered into force on 1 August 2024, requires in Article 10 that the training, validation, and testing data for high-risk AI systems be "relevant, sufficiently representative, and to the best extent possible, free of errors and complete in view of the intended purpose."[21] The Article also requires that data sets account for "the specific geographical, contextual, behavioural or functional setting" in which the system will be used, effectively making the mitigation of coverage bias a legal obligation for developers of AI systems in healthcare, criminal justice, employment, and other high-risk domains.
In the United States, the National Institute of Standards and Technology (NIST) released its AI Risk Management Framework (AI RMF 1.0) on 26 January 2023, which identifies data representativeness as a key factor in managing AI risks.[22] The framework recommends that organizations document the characteristics of their training data, assess potential gaps in coverage, and implement procedures for ongoing monitoring of model performance across relevant subgroups.
The ethical dimensions of coverage bias are significant because the groups most likely to be excluded from training data are often the same groups that have historically been marginalized. Indigenous communities, people with disabilities, elderly populations, low-income households, and speakers of minority languages are all at elevated risk of being underrepresented in datasets. When AI systems trained on biased data are used to make decisions affecting these groups, the result can be a perpetuation or amplification of existing inequalities.