Disparate treatment is the intentional, less favorable treatment of an individual because of a protected attribute such as race, gender, age, religion, national origin, or disability, and in machine learning it describes algorithmic decision systems that explicitly or implicitly rely on those attributes to differentiate between people. It is the computational analog of intentional discrimination in U.S. civil rights law: under Title VII of the Civil Rights Act of 1964, disparate treatment occurs when an individual "is shown to have been singled out and treated less favorably than others similarly situated on the basis of an impermissible criterion," and proving it requires showing the decision was motivated by discriminatory intent.[1][15] Unlike disparate impact, which targets facially neutral policies that produce unequal outcomes regardless of intent, disparate treatment turns on whether a protected characteristic actually drove the decision.[1]

As automated systems increasingly govern hiring, lending, criminal sentencing, healthcare allocation, and housing, disparate treatment has become central to algorithmic fairness. Understanding how machine learning models can perpetuate or introduce intentional discrimination is essential for building equitable AI systems and for ensuring compliance with anti-discrimination law.[11]

ELI5: Explain like I'm 5

Imagine you are picking kids for a soccer team. If you pick players based on how fast they run and how well they kick, that is fair. But if you say "I don't want any kids with red hair on my team" and leave them out just because of their hair color, that is disparate treatment. You are treating some kids differently for a reason that has nothing to do with soccer.

Computers can do the same thing. If a computer program is deciding who gets a job or a loan, and it secretly looks at whether someone is a man or a woman (or uses clues that tell it someone's race), and then gives worse results to one group, that is disparate treatment. The computer is making unfair decisions based on who people are, not on what they can do.

What are the legal foundations of disparate treatment?

Disparate treatment is a legal doctrine rooted in United States anti-discrimination law, particularly Title VII of the Civil Rights Act of 1964. It has since been extended through additional statutes and applied to algorithmic decision-making contexts.[1] The U.S. Equal Employment Opportunity Commission frames the inquiry narrowly: "The issue is whether the employer's actions were motivated by discriminatory intent," which may be shown through direct or circumstantial evidence.[15]

Key statutes

Statute	Year	Scope	Protected characteristics
Civil Rights Act, Title VII	1964	Employment	Race, color, religion, sex, national origin
Age Discrimination in Employment Act (ADEA)	1967	Employment	Age (40 and older)
Fair Housing Act (FHA)	1968	Housing, lending	Race, color, religion, sex, national origin, familial status, disability
Equal Credit Opportunity Act (ECOA)	1974	Credit decisions	Race, color, religion, national origin, sex, marital status, age, public assistance status
Americans with Disabilities Act (ADA)	1990	Employment, public services	Disability
Genetic Information Nondiscrimination Act (GINA)	2008	Employment, health insurance	Genetic information

The McDonnell Douglas burden-shifting framework

The legal standard for proving disparate treatment in cases lacking direct evidence of discrimination was established by the U.S. Supreme Court in McDonnell Douglas Corp. v. Green (1973). The framework consists of three steps:

Prima facie case. The plaintiff must demonstrate that (a) they belong to a protected class, (b) they were qualified for the position or benefit, (c) they suffered an adverse action, and (d) the adverse action occurred under circumstances suggesting discrimination. The burden at this stage is minimal, often described as "de minimis."
Employer's rebuttal. Once the prima facie case is established, the burden shifts to the defendant to articulate a legitimate, nondiscriminatory reason for the adverse action.
Proving pretext. If the defendant provides such a reason, the burden returns to the plaintiff to show that the stated reason is a pretext for discrimination, meaning the true motivation was discriminatory intent.

This framework was designed for human decision-makers, and its application to algorithmic systems presents significant challenges. Machines do not possess intent in the human sense; they execute instructions encoded by their designers. As a result, courts have had to adapt their analysis when evaluating whether an AI system engaged in disparate treatment.[14]

How does disparate treatment differ from disparate impact?

Disparate treatment and disparate impact are the two primary legal theories for addressing discrimination, including in AI contexts. They differ in what must be proven and how liability is established.[1]

Dimension	Disparate treatment	Disparate impact
Core concept	Intentional discrimination based on protected attributes	Neutral practice that disproportionately harms a protected group
Intent required	Yes	No
Key question	Did the decision-maker treat someone differently because of a protected characteristic?	Does the practice produce significantly different outcomes for a protected group?
Burden of proof	Plaintiff must show discriminatory intent	Plaintiff must show statistical disparity; defendant must justify business necessity
AI application	Algorithm explicitly uses or infers protected attributes	Algorithm uses neutral features that correlate with protected attributes
Example	Loan algorithm penalizes applicants from certain zip codes known to correlate with race	Resume screener trained on historically biased data rejects more female applicants
Legal standard	McDonnell Douglas burden-shifting	Griggs v. Duke Power (1971) three-part test

In the context of AI, disparate treatment is often harder to prove than disparate impact because algorithmic systems lack human intent. However, disparate treatment can still be established when a system explicitly incorporates protected attributes, when developers knowingly deploy a discriminatory system, or when proxy variables serve as stand-ins for protected characteristics.[14]

How does disparate treatment arise in machine learning?

Disparate treatment in ML systems can emerge through several mechanisms, ranging from explicit use of protected attributes to more subtle forms of proxy discrimination.[11]

Explicit use of protected attributes

The most straightforward form of disparate treatment occurs when a model directly uses protected attributes (race, gender, age) as input features. For example, a classification model that takes "gender" as an input variable and assigns different scores to male and female applicants engages in disparate treatment by design. While this form is relatively easy to detect and prevent, it still occurs in practice, particularly in legacy systems or when developers are unaware of legal requirements.

Proxy variables and indirect discrimination

Even when protected attributes are excluded from a model's input features, machine learning algorithms can learn to infer them through correlated variables known as proxy variables.[13] Common proxies include:

Zip code or neighborhood: Strongly correlated with race and socioeconomic status due to historical patterns of residential segregation.
Name: First names and surnames can be predictive of gender, race, and ethnicity.
Educational institution: Attending certain schools or universities may correlate with socioeconomic background, race, or gender.
Language patterns: Word choice and phrasing in resumes or applications can correlate with gender or cultural background.
Healthcare spending: Spending levels on medical care are correlated with race due to systemic differences in healthcare access.
Purchase history and browsing behavior: Consumer data can reveal gender, age, and other protected characteristics.

Proxy discrimination is especially challenging because removing the most obvious proxies does not solve the problem. Machine learning models, particularly deep learning systems, can discover non-linear combinations of seemingly neutral features that reconstruct protected attributes with high accuracy. Simply denying a model access to intuitive proxies causes it to locate less intuitive ones.[13]

Biased training data

When training data reflects historical patterns of discrimination, models learn to replicate those patterns.[1] If a hiring model is trained on a dataset of past hiring decisions where certain demographic groups were systematically disadvantaged, the model will learn to associate characteristics of those groups with negative outcomes. This can constitute disparate treatment when the model effectively learns to use protected attributes (directly or through proxies) as a basis for its predictions.

Pretrained model bias

Large pretrained models such as language models and vision models can encode societal stereotypes from their training corpora. When these models are used as components in downstream decision-making systems (for example, as embedding layers in a resume screening tool), the encoded biases can propagate into final decisions. Research has shown that word embeddings trained on large text corpora associate certain professions with specific genders and associate negative attributes with particular racial groups.[11]

Feedback loops

Disparate treatment can be amplified through feedback loops. If a biased model denies opportunities to members of a protected group, the resulting data (showing worse outcomes for that group) reinforces the model's discriminatory patterns in subsequent retraining cycles. Over time, the bias compounds, creating a self-reinforcing cycle of discrimination.[11]

Real-world case studies

Several high-profile cases have demonstrated how disparate treatment manifests in deployed AI systems.

Amazon's resume screening tool

In 2014, Amazon developed an automated resume screening system to evaluate job applicants. The system was trained on resumes submitted to the company over the preceding decade. Because Amazon's engineering workforce was predominantly male, the model learned to penalize resumes containing indicators of female applicants. Specifically, the algorithm downgraded resumes that included the word "women's" (as in "women's chess club captain") and penalized graduates of certain all-women's colleges. It also favored verbs like "executed" and "captured," which appeared more frequently on male engineers' resumes. Amazon attempted to correct the bias but ultimately concluded that the system could not be prevented from finding new proxies for gender, and the project was abandoned in 2017.[7]

Meta's housing ad targeting

In June 2022, the U.S. Department of Justice reached a settlement with Meta Platforms (formerly Facebook) over allegations that Meta's housing ad delivery system violated the Fair Housing Act.[8] The DOJ alleged that Meta's machine learning algorithms used protected characteristics, including race, color, religion, sex, disability, familial status, and national origin, to determine which subset of an advertiser's audience would actually receive housing advertisements. Under the settlement, Meta was required to pay the maximum penalty of $115,054 under the FHA and to develop a new ad delivery system that did not rely on protected characteristics. An independent third-party reviewer was appointed to verify compliance.[8]

COMPAS recidivism prediction

The Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) system, developed by Northpointe (now Equivant), is a risk assessment tool used in 46 U.S. states to predict whether criminal defendants are likely to reoffend. In 2016, ProPublica analyzed over 10,000 criminal defendants in Broward County, Florida, and found that Black defendants were 77% more likely to be falsely labeled as high risk for violent recidivism and 45% more likely to be incorrectly flagged for any future crime. White defendants, conversely, were more likely to be falsely labeled as low risk.[3]

Northpointe countered that COMPAS was calibrated: the overall accuracy rate was approximately 60% for both Black and white defendants.[5] This dispute highlighted a fundamental tension in algorithmic fairness, often called the "impossibility of fairness" theorem: calibration, equal false positive rates, and equal false negative rates cannot all be satisfied simultaneously when base rates differ between groups.[6]

Optum healthcare algorithm

A 2019 study published in Science by Obermeyer et al. revealed that a healthcare risk prediction algorithm developed by Optum and used by hospital systems across the United States systematically underestimated the healthcare needs of Black patients.[4] The algorithm used healthcare costs as a proxy for health needs, but because Black patients historically spent less on healthcare (due to unequal access, not better health), the system assigned them lower risk scores. The study estimated that this bias affected approximately 200 million people annually. After the study, a revised algorithm that incorporated direct health predictions alongside cost data reduced the bias by 84%.[4]

Workday AI hiring tools

In July 2024, Judge Rita Lin of the U.S. District Court for the Northern District of California ruled in Mobley v. Workday that AI vendors can be held directly liable for employment discrimination under an "agent" theory. The plaintiff alleged that Workday's AI-powered applicant screening tools discriminated on the basis of race, age, and disability after he was rejected from more than 100 jobs. The court dismissed the disparate treatment (intentional discrimination) claim for insufficient evidence of discriminatory intent but allowed disparate impact claims under Title VII, the ADEA, and the ADA to proceed.[16] This ruling was significant because it established that third-party AI vendors, not just employers, can face liability for discriminatory outcomes produced by their tools. In May 2025, the same court granted conditional certification of a nationwide ADEA collective action, allowing applicants aged 40 and older who were rejected through Workday's screening tools to join the case.[16]

What fairness concepts relate to disparate treatment?

The machine learning fairness literature has developed several formal concepts that relate directly to disparate treatment.

Fairness through unawareness

Fairness through unawareness (FTU) is the simplest approach to preventing disparate treatment: exclude protected attributes from the model's input features. In principle, if the model never "sees" race or gender, it cannot discriminate based on those characteristics. However, FTU is widely regarded as insufficient because models can infer protected attributes from proxy variables. Even when sensitive attributes are removed, algorithmic models may still learn such information through complex non-linear relationships in the data, perpetuating or amplifying systemic bias.[11] Dwork et al. (2012) demonstrate that "fairness through blindness" fails to rule out abuses such as self-fulfilling prophecies, where members of a protected group are deliberately mishandled in ways that statistical parity does not detect.[2]

Fairness through awareness

Fairness through awareness (FTA), proposed by Dwork et al. (2012), takes the opposite approach.[2] Rather than ignoring protected attributes, FTA explicitly accounts for them. The paper's central principle is "a (hypothetical) task-specific metric for determining the degree to which individuals are similar with respect to the classification task at hand" combined with "an algorithm for maximizing utility subject to the fairness constraint, that similar individuals are treated similarly."[2] This directly operationalizes the legal standard of treating "similarly situated individuals" equivalently.[2]

Individual fairness

Individual fairness requires that any two individuals who are similar with respect to a given task should receive similar predictions from the model.[2] This concept is closely aligned with the legal notion of disparate treatment, which focuses on whether specific individuals were treated differently because of their protected attributes. The challenge lies in defining a meaningful similarity metric that captures task-relevant characteristics without incorporating protected attributes.

Group fairness metrics

Group fairness metrics evaluate whether a model's outcomes are equitable across demographic groups. While these metrics are more commonly associated with disparate impact analysis, they can also help detect patterns consistent with disparate treatment.

Metric	Definition	Connection to disparate treatment
Demographic parity	Equal selection rates across groups	Violations may indicate group-level differential treatment
Equalized odds	Equal true positive and false positive rates across groups	Unequal error rates may suggest the model treats groups differently
Calibration	Predicted probabilities reflect actual outcomes equally across groups	Miscalibration across groups may reflect biased feature reliance
Predictive parity	Equal positive predictive values across groups	Differences may indicate the model is less reliable for certain groups

A key theoretical result is the impossibility theorem: demographic parity, equalized odds, and calibration cannot all be satisfied simultaneously except in trivial cases (a perfect predictor or equal base rates across groups).[6] This means practitioners must choose which fairness criteria to prioritize based on the specific application and its legal and ethical requirements.

How is disparate treatment detected and mitigated?

Researchers and practitioners have developed a range of techniques to detect, measure, and mitigate disparate treatment in machine learning systems. These methods are typically categorized by when they are applied in the ML pipeline.[10]

Pre-processing methods

Pre-processing techniques modify the training data before model training to reduce bias.

Resampling. Oversampling underrepresented groups or undersampling overrepresented groups to balance the training data across protected attributes.
Reweighting. Assigning different weights to training examples so that the model treats all groups as equally important during optimization.
Data transformation. Modifying feature values to remove correlations between features and protected attributes while preserving predictive utility.
Label correction. Identifying and correcting labels in the training data that reflect historical discrimination rather than true merit or risk.

In-processing methods

In-processing techniques incorporate fairness directly into the model training process.

Fairness-constrained optimization. Adding fairness constraints (such as demographic parity or equalized odds) to the model's loss function. The model is then optimized to maximize predictive performance subject to the fairness constraints.
Adversarial debiasing. Training two neural networks simultaneously: a predictor that performs the main task and an adversary that tries to predict protected attributes from the predictor's outputs. The predictor is trained to maximize accuracy while minimizing the adversary's ability to infer protected attributes. This technique is effective for achieving demographic parity and conditional demographic parity but requires careful tuning to ensure convergence.[9]
Regularization-based approaches. Adding regularization terms to the loss function that penalize the model for relying on features correlated with protected attributes.
Causal modeling. Using causal inference techniques to distinguish between legitimate and discriminatory pathways from inputs to outputs, blocking only the discriminatory causal paths.

Post-processing methods

Post-processing techniques adjust the model's outputs after training to achieve fairness.[10]

Threshold adjustment. Setting different classification thresholds for different demographic groups to equalize error rates or selection rates.
Calibration. Adjusting predicted probabilities so that they are equally accurate across demographic groups.
Reject option classification. Reassigning predictions near the decision boundary to favor disadvantaged groups, under the reasoning that uncertain predictions are most likely to be affected by bias.

Detection and auditing

Counterfactual testing. Changing the protected attribute (or its proxies) in an individual's data while holding other features constant, then observing whether the model's output changes. A change in the prediction suggests the model is relying on protected attributes.
Feature importance analysis. Using interpretability techniques (such as SHAP values, LIME, or permutation importance) to assess whether protected attributes or their proxies are influential in the model's decisions.
Statistical disparity analysis. Comparing model outputs across demographic groups using metrics such as selection rate ratios, false positive rate differences, or precision and recall disparities.
Audit testing. Submitting matched pairs of inputs (identical except for protected attributes) to a deployed system and comparing the outputs, analogous to housing or employment discrimination testing in civil rights enforcement.

What does the regulatory landscape look like?

Governments and regulatory bodies around the world have begun addressing algorithmic discrimination through legislation and guidance.

United States

The U.S. relies primarily on existing civil rights statutes (Title VII, FHA, ECOA, ADA, ADEA) to address algorithmic discrimination. Key regulatory developments include:

EEOC guidance (2023). The Equal Employment Opportunity Commission issued guidance clarifying that employers can be held liable under Title VII for discriminatory outcomes produced by AI hiring tools, whether developed in-house or purchased from third-party vendors.
CFPB guidance on adverse action notices. The Consumer Financial Protection Bureau has stated that creditors using AI or complex algorithms must still provide specific and accurate reasons when denying credit, as required by ECOA and Regulation B. "Black box" models that cannot generate explanations do not excuse non-compliance.
State and local laws. New York City's Local Law 144 (effective July 5, 2023) requires employers using automated employment decision tools to conduct annual bias audits and publish summary results. Illinois and Maryland have enacted restrictions on AI-based video interview analysis.

European Union

The EU AI Act, which entered into force on August 1, 2024, with full applicability by August 2, 2026, establishes the world's first comprehensive legal framework for AI regulation.[12]

Risk-based classification. AI systems are categorized into four risk levels: unacceptable, high, limited, and minimal. Systems used in employment, education, law enforcement, and access to essential services are classified as high risk.
Bias detection requirements. Article 10(5) of the AI Act permits processing special categories of personal data (such as race or ethnicity) strictly for the purpose of bias monitoring, detection, and correction in high-risk AI systems, subject to appropriate safeguards.[12]
Prohibition of discriminatory practices. The Act prohibits AI systems that exploit vulnerabilities based on age, disability, or social or economic situation, and systems that perform social scoring leading to detrimental treatment.
Interaction with GDPR. The General Data Protection Regulation's restrictions on processing special categories of personal data create some tension with the AI Act's provisions for bias detection, since bias audits may require access to the very data that GDPR restricts.

Other jurisdictions

Canada's Artificial Intelligence and Data Act (AIDA), proposed as part of Bill C-27, would require impact assessments for high-impact AI systems and impose obligations to mitigate risks of bias. Brazil, China, and several other countries have also introduced or proposed AI-specific legislation that addresses algorithmic discrimination.

Why is disparate treatment hard to apply to AI?

Several conceptual and practical challenges complicate the application of the disparate treatment framework to machine learning systems.

The intent problem

Traditional disparate treatment law requires proof of discriminatory intent. Machines do not have intent; they execute mathematical operations on data. This creates a fundamental mismatch between the legal doctrine and the technology it must regulate. Courts have begun adapting by looking at the intent of the system's designers and operators rather than the system itself. If a company knows that its AI system produces discriminatory outcomes and continues to use it, courts may infer discriminatory intent.[14]

The opacity problem

Many modern ML models, particularly deep neural networks, operate as "black boxes" whose internal decision-making processes are difficult to interpret. This opacity makes it hard to determine whether a model is relying on protected attributes or their proxies. While interpretability techniques can provide some insight, they often offer approximate rather than definitive explanations of model behavior.

The proxy problem

Proxy variables are pervasive in real-world data. Because many features in a typical dataset are correlated with protected attributes, it is extremely difficult to construct a model that is both accurate and completely free from proxy discrimination. Removing known proxies simply causes models to find less obvious ones through complex feature interactions. This creates a fundamental tension between predictive accuracy and fairness.[13]

Competing fairness definitions

As demonstrated by the COMPAS case, different mathematical definitions of fairness can conflict with one another.[5] A model that is calibrated across groups may still have unequal false positive rates. A model that achieves demographic parity may sacrifice calibration. Practitioners must make value judgments about which fairness criteria to prioritize, and those judgments may differ depending on the application domain, the stakes involved, and the legal requirements that apply.

Intersectionality

Individuals belong to multiple protected groups simultaneously (for example, a person who is both Black and female). Disparate treatment analyses that consider only one protected attribute at a time may miss discrimination that affects intersectional groups. A model may treat Black applicants fairly on average and treat female applicants fairly on average, while still systematically disadvantaging Black women. Detecting and mitigating intersectional discrimination requires examining outcomes across combinations of protected attributes, which increases the complexity of fairness analysis considerably.

How can organizations prevent disparate treatment?

Organizations developing or deploying AI systems can take several steps to reduce the risk of disparate treatment.

Conduct thorough data audits. Before training, examine the dataset for historical biases, label quality, and correlations between features and protected attributes.
Document model design decisions. Maintain detailed records of feature engineering choices, training procedures, and fairness evaluations. Model cards and datasheets for datasets can standardize this documentation.
Test for proxy discrimination. Use counterfactual testing and feature importance analysis to identify whether the model relies on proxies for protected attributes.
Apply appropriate mitigation techniques. Select pre-processing, in-processing, or post-processing methods based on the specific fairness requirements and the nature of the application.
Monitor deployed systems continuously. Track model outputs across demographic groups over time to detect emerging disparities or drift in fairness metrics.
Establish human oversight. Maintain human review processes for high-stakes decisions, ensuring that automated predictions are not blindly followed.
Engage diverse teams. Include individuals from different backgrounds in the design, testing, and oversight of AI systems to surface blind spots and implicit assumptions.
Comply with applicable regulations. Stay informed about evolving legal requirements, including adverse action notice obligations, bias audit mandates, and sector-specific fairness rules.

See also

References

Barocas, S. and Selbst, A.D. (2016). "Big Data's Disparate Impact." California Law Review, 104(3), 671-732.
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. (2012). "Fairness Through Awareness." Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (ITCS), 214-226. arXiv:1104.3913.
Angwin, J., Larson, J., Mattu, S., and Kirchner, L. (2016). "Machine Bias: There's Software Used Across the Country to Predict Future Criminals. And It's Biased Against Blacks." ProPublica, May 23, 2016.
Obermeyer, Z., Powers, B., Vogeli, C., and Mullainathan, S. (2019). "Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations." Science, 366(6464), 447-453.
Chouldechova, A. (2017). "Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments." Big Data, 5(2), 153-163.
Kleinberg, J., Mullainathan, S., and Raghavan, M. (2016). "Inherent Trade-Offs in the Fair Determination of Risk Scores." arXiv preprint arXiv:1609.05807.
Dastin, J. (2018). "Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women." Reuters, October 10, 2018.
U.S. Department of Justice (2022). "United States v. Meta Platforms, Inc." Civil Rights Division, Fair Housing Act Enforcement.
Zhang, B.H., Lemoine, B., and Mitchell, M. (2018). "Mitigating Unwanted Biases with Adversarial Learning." Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, 335-340.
Chen, V., Hooker, S., Padilla, C., and Ritter, S. (2023). "Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey." ACM Journal on Responsible Computing, 1(1), 1-33.
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., and Galstyan, A. (2021). "A Survey on Bias and Fairness in Machine Learning." ACM Computing Surveys, 54(6), 1-35.
European Parliament and Council of the European Union (2024). "Regulation (EU) 2024/1689: Artificial Intelligence Act." Official Journal of the European Union.
Tschantz, M.C. (2022). "What is Proxy Discrimination?" Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT), 1993-2003.
Kim, P.T. (2022). "Race-Aware Algorithms: Fairness, Nondiscrimination and Affirmative Action." California Law Review, 110(5), 1539-1596.
U.S. Equal Employment Opportunity Commission. "CM-604 Theories of Discrimination." EEOC Compliance Manual.
Mobley v. Workday, Inc., No. 3:23-cv-00770 (N.D. Cal.), order on motion to dismiss (July 12, 2024) and order granting conditional certification of ADEA collective action (May 16, 2025).

Editing: Disparate Treatment