# Logistic Regression

> Source: https://aiwiki.ai/wiki/logistic_regression
> Updated: 2026-07-29
> Fact-checked: 2026-07-29
> Categories: Machine Learning, Statistics
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Logistic Regression." aiwiki.ai, 29 Jul 2026. https://aiwiki.ai/wiki/logistic_regression
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**Logistic regression** is a statistical model for a binary response. It represents the conditional probability of one outcome as the logistic transformation of a linear predictor. In [machine learning](https://aiwiki.ai/wiki/machine_learning), the same model is used for [supervised learning](https://aiwiki.ai/wiki/supervised_learning) and [binary classification](https://aiwiki.ai/wiki/binary_classification). The fitted model produces probabilities. Turning those probabilities into actions or class labels is a separate decision step, so a threshold of 0.5 is a software convention rather than part of the definition of logistic regression.[1][2]

The model is linear in its coefficients and in the log-odds, but not in the predicted probability. This distinction gives logistic regression a useful combination of interpretable parameters, bounded probability estimates, and a convex unpenalized loss. Those properties do not make a fitted model automatically valid or calibrated. Its performance still depends on the data-generating process, feature specification, sampling design, regularization, and evaluation procedure.[1][3]

## History and terminology

The logistic function predates logistic regression. Pierre-Francois Verhulst introduced a logistic growth curve in the nineteenth century, while later work brought the curve into bioassay and statistics. Joseph Berkson introduced the word "logit" in a 1944 comparison of logistic and probit analysis. David Cox's 1958 paper developed regression methods for sequences of binary outcomes and reviewed earlier work rather than claiming a single point of invention. In 1972, John Nelder and Robert Wedderburn placed binomial models with a logit link in the broader framework of [generalized linear models](https://aiwiki.ai/wiki/generalized_linear_model).[4][5][6][7]

These developments should be kept separate:

| Development | What it established |
|---|---|
| Logistic curve | A bounded S-shaped mathematical function, initially used for growth |
| Logit | The natural logarithm of the odds, named by Berkson |
| Binary regression | A model relating binary outcome probabilities to explanatory variables |
| Generalized linear model formulation | A binomial response distribution, linear predictor, and link function treated within one framework |

No single paper created every component of modern logistic regression. Assigning the entire method to Cox's 1958 paper omits the earlier logistic and logit work, while saying that Berkson's 1944 paper already contained every modern formulation is also too broad.[4][5][6]

## Binary logistic model

Let the response for observation \(i\) be \(Y_i \in \{0,1\}\), and let

\[
p_i = P(Y_i=1 \mid \mathbf{x}_i).
\]

The odds of the event are \(p_i/(1-p_i)\). The [log-odds](https://aiwiki.ai/wiki/log-odds), or logit, are

\[
\operatorname{logit}(p_i)
= \log\left(\frac{p_i}{1-p_i}\right).
\]

Binary logistic regression specifies

\[
\operatorname{logit}(p_i)
= \eta_i
= \beta_0 + \beta_1x_{i1}+\cdots+\beta_px_{ip}.
\]

Applying the inverse-logit, also called the logistic or [sigmoid function](https://aiwiki.ai/wiki/sigmoid_function), gives

\[
p_i
= \sigma(\eta_i)
= \frac{1}{1+\exp(-\eta_i)}.
\]

For finite \(\eta_i\), the result lies strictly between 0 and 1. The complement is \(P(Y_i=0 \mid \mathbf{x}_i)=1-p_i\). The model therefore supplies a coherent pair of conditional probabilities for a binary response.[1][3]

### What "linear" means

The model assumes that the logit is additive and linear in the included coefficient terms. It does not require the probability itself to be a straight-line function of every original measurement. Predictors may include:

| Term in the design matrix | Purpose |
|---|---|
| Continuous predictor \(x\) | Represents a constant change in log-odds per unit of \(x\) |
| Indicator variables | Encode categories relative to a documented reference level |
| Interaction \(x_1x_2\) | Allows one predictor's coefficient to depend on another |
| Polynomial or spline basis | Allows a curved relationship between an original predictor and the logit |
| Offset | Adds a known term whose coefficient is fixed at one in implementations that support it |

This flexibility does not remove the need to specify the terms carefully. A linear logit in age, for example, is a different model from a logit containing an age spline. Arbitrarily categorizing a continuous predictor can discard information and create results that depend on chosen cut points. Transformations and interactions should be motivated before inspecting the final test results whenever the goal is statistical inference.[3][8]

### Probability estimation and class decisions

Logistic regression estimates \(p_i\). A classifier then maps \(p_i\) to a label, commonly by predicting class 1 when \(p_i \ge t\) for a threshold \(t\). At \(t=0.5\), the corresponding boundary is

\[
\beta_0+\beta_1x_1+\cdots+\beta_px_p=0.
\]

This is a hyperplane in the constructed feature space. If that space includes splines, polynomials, or interactions, its projection back into the original variables can be curved.

The threshold is an operating decision. It may depend on the relative consequences of false positives and false negatives, resource constraints, class prevalence, or an explicit utility function. Scikit-learn's binary `predict` convention uses 0.5 for probability estimates or zero for decision scores, but its documentation treats probability estimation and action selection as distinct problems and provides tools for tuning or fixing the threshold.[2] Threshold selection should use validation data or a prespecified decision rule, not the same observations used to report final performance.

## Likelihood and estimation

Under the basic model, each response is conditionally Bernoulli with probability \(p_i\). For observations treated as independent, the likelihood is

\[
L(\boldsymbol{\beta})
= \prod_{i=1}^{n}
p_i^{y_i}(1-p_i)^{1-y_i}.
\]

Its log is

\[
\ell(\boldsymbol{\beta})
= \sum_{i=1}^{n}
\left[y_i\log(p_i)+(1-y_i)\log(1-p_i)\right].
\]

Maximum likelihood estimation chooses coefficients that maximize \(\ell\), or equivalently minimize the negative log-likelihood. Dividing that objective by \(n\) produces the mean binary log loss, also called binary [cross-entropy](https://aiwiki.ai/wiki/cross-entropy). The scaling changes the numeric objective but not an unpenalized optimum.[1][3]

For a design matrix \(X\), outcome vector \(\mathbf{y}\), and fitted probability vector \(\mathbf{p}\), the score is

\[
\nabla \ell(\boldsymbol{\beta})=X^\mathsf{T}(\mathbf{y}-\mathbf{p}).
\]

The negative log-likelihood Hessian is

\[
X^\mathsf{T}WX,
\]

where \(W\) is diagonal with entries \(p_i(1-p_i)\). This matrix is positive semidefinite, so the unpenalized negative log-likelihood is convex. Strict convexity and a unique finite estimate require additional conditions, including adequate rank and the absence of separation. Convexity alone does not guarantee that an unpenalized finite minimizer exists.[1][9]

### Numerical optimization

Except for special cases, the likelihood equations do not yield a closed-form coefficient estimate. Software uses iterative algorithms:

| Method | Main idea | Practical consideration |
|---|---|---|
| Newton-Raphson or Fisher scoring | Uses first and second derivative information | Leads to iteratively weighted least squares for standard generalized linear models |
| Limited-memory BFGS | Approximates curvature without storing a full Hessian | A common general-purpose choice |
| Newton-CG or Newton-Cholesky | Uses curvature more directly | Memory and runtime depend strongly on the number of parameters |
| SAG or SAGA | Uses incremental gradient information | Useful for large data; scaling affects convergence |
| Coordinate descent | Updates coefficients by coordinate | Effective for regularized paths, especially with nonsmooth L1 penalties |
| [Stochastic gradient descent](https://aiwiki.ai/wiki/stochastic_gradient_descent_sgd) | Uses individual observations or mini-batches | Supports streaming and very large sparse problems, but requires optimization choices |

Nelder and Wedderburn described maximum-likelihood fitting for generalized linear models through iteratively weighted linear regression.[7] Coordinate-descent paths for L1, L2, and elastic-net generalized linear models are implemented by `glmnet`, while LIBLINEAR provides large-scale linear classification algorithms including logistic regression.[10][11] An optimizer reporting convergence means that it satisfied its stopping rule. It does not establish that the model is correctly specified, stable, or useful on new data.

Feature scales can materially affect numerical conditioning and the meaning of a common penalty. Scaling is particularly important for scale-sensitive solvers and when penalty coefficients are to be compared. Scikit-learn specifically notes that fast convergence of SAG and SAGA is guaranteed only when features are on approximately the same scale.[12]

### Grouped binomial observations and weights

Binary records may be represented individually, or repeated trials with the same covariates may be grouped as successes out of a known number of trials. Both representations can encode the same binomial likelihood when their weights and trial totals are specified correctly.[3][26] Observation weights can instead mean frequency weights, sampling weights, or importance weights, depending on the software. Those interpretations are not interchangeable. A report should state what a weight represents and how uncertainty was calculated.

## Coefficient interpretation

For a continuous predictor \(x_j\), with all other design terms held fixed, a one-unit increase changes the log-odds by \(\beta_j\). Exponentiation yields the conditional odds ratio

\[
\operatorname{OR}_j=\exp(\beta_j).
\]

If \(\beta_j=0.7\), then \(\exp(0.7)\) is about 2.01. Under a model without an interaction involving \(x_j\), the fitted odds are therefore multiplied by about 2.01 for a one-unit increase in \(x_j\), conditional on the other included predictors. This does not mean that the probability doubles. Odds and probabilities are different quantities.

Interpretation depends on coding:

| Modeling choice | Effect on interpretation |
|---|---|
| Unit of a continuous predictor | Determines the change represented by one coefficient unit |
| Centering | Changes the covariate values at which the intercept and interactions are interpreted |
| Standardization | Expresses a coefficient per chosen scale unit, often one sample standard deviation |
| Reference category | Determines which category an indicator coefficient compares against |
| Interaction | Makes a main-effect coefficient conditional on the interacting variable's reference value |
| Nonlinear basis | Means no single coefficient describes the complete effect of the original predictor |

The intercept \(\beta_0\) is the log-odds when every numeric predictor equals zero and every categorical predictor is at its reference level. That combination may be outside the observed data or scientifically meaningless, which is one reason to center predictors deliberately.

### Probability changes and marginal effects

For a continuous predictor that enters only as \(\beta_jx_j\),

\[
\frac{\partial p_i}{\partial x_{ij}}
= \beta_jp_i(1-p_i).
\]

The probability change depends on the observation's other predictors through \(p_i\). It is largest in magnitude near \(p_i=0.5\) and smaller near zero or one. For a binary or categorical predictor, a discrete change in predicted probability is usually clearer than a derivative.

Common probability-scale summaries include:

| Summary | Calculation | Interpretation |
|---|---|---|
| Marginal effect at specified values | Change or derivative at a documented covariate profile | Applies to that profile |
| Average marginal effect | Compute each observation's effect, then average | Average over the selected sample's covariate distribution |
| Adjusted prediction | Average predicted probabilities after setting a predictor to a specified value | A standardized probability contrast |

Average marginal effects should identify the data over which averaging occurred. Norton, Dowd, and Maciejewski describe these summaries and distinguish them from odds-ratio interpretation.[13]

### Conditional association, noncollapsibility, and causality

Logistic coefficients are conditional on the predictors in the fitted model. Odds ratios are noncollapsible: even without confounding, a conditional odds ratio can differ from a marginal odds ratio after adding or removing a prognostic covariate. A coefficient changing across two logistic models is therefore not, by itself, proof of confounding or mediation.[14]

Likewise, including a predictor in a logistic regression does not make its coefficient causal. Causal interpretation requires an appropriate study design and assumptions about treatment assignment, confounding, selection, measurement, interference, and model specification. Conditioning on a collider or an affected post-treatment variable can introduce bias. The model's algebra supplies an association measure; the research design determines which causal interpretation, if any, is warranted.

## Regularization and bias reduction

[Regularization](https://aiwiki.ai/wiki/regularization) modifies the fitting objective by penalizing coefficients. One common elastic-net form is

\[
-\ell(\boldsymbol{\beta})
+ \lambda\left[
\alpha\sum_{j=1}^{p}|\beta_j|
+ \frac{1-\alpha}{2}\sum_{j=1}^{p}\beta_j^2
\right],
\]

usually excluding the intercept, although software conventions vary. Here \(\lambda\) controls overall strength and \(\alpha\) blends L1 and L2 penalties.

| Penalty | Typical effect | Important qualification |
|---|---|---|
| L2 | Shrinks correlated and weakly identified coefficients toward zero | Coefficients generally remain nonzero |
| L1 | Can set coefficients exactly to zero | Selection can be unstable among correlated predictors |
| Elastic net | Combines shrinkage with possible sparsity | Requires both overall strength and mixing choices |
| No predictive penalty | Retains the ordinary likelihood | Can be unstable or lack a finite estimate under separation |

L1 sparsity can aid compression or [feature selection](https://aiwiki.ai/wiki/feature_selection), but a selected set is not automatically the set of true causes. In high-dimensional settings, L1 and L2 can have different sample-complexity behavior, and elastic-net paths provide a computationally efficient continuum between them.[10][15] Hyperparameters should be selected within the validation procedure. If the same folds are used to select the penalty and to announce final performance, the resulting estimate can be optimistic.

Regularized coefficients are biased by design. Standard errors and \(p\)-values from an ordinary unpenalized maximum-likelihood fit do not automatically apply after data-dependent penalty tuning or feature selection. Predictive modeling and classical inferential modeling may therefore use different workflows even when both are called logistic regression.

Firth's bias-reduction method is a separate likelihood-based adjustment. For canonical exponential-family models it corresponds to a Jeffreys-prior penalty and removes the first-order term in the asymptotic bias. It is often useful when ordinary logistic maximum likelihood is affected by separation or small-sample bias.[16] It should not be described as identical to ordinary L2 regularization, because its penalty and inferential purpose differ.

## Model conditions and failure modes

Logistic regression does not require predictors to follow a multivariate normal distribution. It does, however, impose assumptions on the conditional response model and on how the data enter the likelihood.

### Specification and dependence

The basic likelihood treats responses as conditionally independent given the modeled predictors. Repeated measures, clustered sampling, matched sets, or spatial dependence can make ordinary model-based standard errors incorrect. Depending on the estimand and design, alternatives include cluster-robust covariance estimates, generalized estimating equations, mixed-effects logistic regression, conditional logistic regression, or an explicit dependence model.[8]

The conditional mean must also be adequately specified. Omitted nonlinear terms or interactions can distort both predictions and coefficient interpretations. Domain knowledge, prespecified flexible terms, residual plots, and validation across relevant covariate ranges are more informative than a claim that logistic regression has "no assumptions."[8]

Exact linear dependence in the design matrix prevents separate identification of all included coefficients. Near-collinearity can produce large uncertainty and sensitivity even when software returns a solution. Scaling a variable changes a coefficient's units but does not cure substantive redundancy.

### Complete and quasi-complete separation

Separation occurs when a linear combination of predictors perfectly distinguishes observed outcomes. Under complete separation, the ordinary likelihood can keep improving as one or more coefficient magnitudes diverge. Under quasi-complete separation, some fitted groups overlap at the boundary but a finite ordinary estimate can still fail to exist. Albert and Anderson gave conditions distinguishing complete separation, quasi-complete separation, and overlap.[9]

Warning signs include:

| Warning sign | Why it matters |
|---|---|
| Extremely large coefficients or standard errors | The likelihood may be nearly flat toward an infinite estimate |
| Predicted probabilities numerically near zero or one for all training cases | The sample may be separable or nearly separable |
| Failure to converge despite more iterations | More iterations cannot create a finite maximum when none exists |
| Sparse cross-tabulations for categorical levels | A level may contain only one observed outcome |
| A solver's "perfect prediction" warning | The implementation has detected a separation pattern |

Possible responses include verifying the data and coding, combining categories only when scientifically justified, collecting more informative observations, using a documented penalized approach, or using Firth bias reduction. Silently increasing the iteration limit is not a principled solution to separation.

### Sample size and events per variable

A historical simulation study found problems in some settings when there were fewer than about ten outcome events per candidate predictor variable, which helped popularize a "ten events per variable" rule.[17] That rule is not a universal requirement. Later simulation work found no rationale for a single cutoff, because performance also depends on total sample size, event fraction, coefficient sizes, predictor distributions, correlations, and separation.[18]

Modern sample-size planning for prediction models considers the number of candidate parameter terms, anticipated outcome proportion, expected model fit, shrinkage, and the precision of the overall risk estimate. A factor with four levels consumes three parameters, and a spline or interaction consumes more than one, so counting named variables alone is insufficient.[19] The appropriate calculation also differs between estimating a descriptive association, building a prediction model, externally validating a model, and estimating a treatment effect.

### Rare outcomes and sampling

A rare positive class is not itself proof that logistic regression is invalid. It can, however, produce few informative events, unstable estimates, poor probability resolution, and misleading threshold metrics. Case-control sampling can be efficient for estimating certain slope associations, but the fitted intercept and raw predicted probabilities do not generally recover population prevalence without accounting for the sampling design. Methods for rare-event settings address finite-sample bias and probability correction, but they do not replace careful design or external validation.[20]

### Missing data, leakage, and transport

Complete-case fitting is valid only under restrictive missingness conditions and can waste information. Imputation, missingness indicators, weighting, or explicit missing-data models should match the scientific setting. Preprocessing that learns from data, including imputation, scaling, encoding, feature selection, and penalty selection, must be fitted inside each training fold to prevent leakage.[8][19]

Prediction outside the observed covariate range relies on extrapolating the linear predictor. A model can also lose calibration when prevalence, measurement, treatment practice, or the relationship between predictors and outcome changes. Evaluation on data separated by time, site, or population is more informative than a random split when deployment will cross those boundaries.[8][21]

## Diagnostics and evaluation

Evaluation should match the model's intended use. Coefficient inference, probability estimation, ranking, and binary action selection are related but distinct tasks.

### Fit and influence

Useful checks include convergence diagnostics, coefficient stability, design-matrix rank, response counts by important categorical levels, residual patterns, leverage, and influence. Deviance and Pearson residuals can identify observations poorly represented by the fitted mean, while leverage and case-deletion measures assess sensitivity. A surprising observation is not automatically an error; investigation should precede exclusion.[8]

Likelihood-ratio comparisons can assess nested unpenalized models under their regularity conditions. Wald tests may be unreliable with small samples, large coefficients, or separation. A nonsignificant coefficient is not evidence that the predictor has exactly no association, and a small \(p\)-value does not establish practical importance, calibration, or causal validity.[8][9]

A single goodness-of-fit test cannot certify a model. Such tests can have little power in small samples and can flag minor deviations in large samples. Graphical checks and out-of-sample behavior should accompany, rather than be replaced by, one test statistic.[8][21]

### Probability accuracy and calibration

Log loss evaluates the probability assigned to the observed outcome and penalizes confident errors strongly. The Brier score is the mean squared error of binary probability forecasts. Both depend on the full probability estimate, not only a chosen class label.[21]

Calibration asks whether predicted probabilities agree with observed outcome frequencies. A model with predictions near 0.20 should, in an appropriate group of comparable cases, see the outcome occur about 20 percent of the time. Calibration can be summarized at several levels:

| Diagnostic | Question |
|---|---|
| Calibration-in-the-large | Are predictions systematically too high or too low overall? |
| Calibration slope | Are predictions too extreme or not extreme enough? |
| Calibration curve | Does agreement vary across the probability range? |
| Binned reliability plot | How do grouped observed frequencies compare with grouped predictions? |

Van Calster and colleagues distinguish calibration from discrimination and recommend examining the full calibration hierarchy, including a flexible calibration curve.[21] Logistic regression is sometimes well calibrated when its logit structure is appropriate and regularization is suitable, but calibration is not automatic. Scikit-learn's own example makes those conditions explicit.[22] Calibration should be assessed on data not used to fit the displayed curve, with uncertainty and sample size considered.

### Discrimination and ranking

A [ROC curve](https://aiwiki.ai/wiki/roc_receiver_operating_characteristic_curve) plots sensitivity against false-positive rate as a score threshold varies. Its area under the curve measures ranking discrimination: the probability that a randomly selected positive case receives a higher score than a randomly selected negative case, with conventions for ties. It does not measure probability calibration and does not select an operating threshold.[23]

For a strongly [class-imbalanced dataset](https://aiwiki.ai/wiki/class-imbalanced_dataset), precision-recall curves can reveal changes in positive-class retrieval that are visually muted in ROC space. Precision depends on outcome prevalence, so comparisons across datasets with different prevalences require care.[24] Neither ROC AUC nor average precision captures the actual costs of downstream actions.

### Threshold-dependent measures

After a threshold is fixed, a [confusion matrix](https://aiwiki.ai/wiki/confusion_matrix) supplies true-positive, false-positive, true-negative, and false-negative counts. Accuracy, sensitivity, specificity, precision, negative predictive value, and F-scores are functions of those counts. They should be reported with the positive class, threshold, evaluation population, and uncertainty.

The best threshold for one objective need not be best for another. A threshold can be selected from explicit utilities or constraints, such as a minimum sensitivity, instead of maximizing a generic metric. If a threshold is tuned on validation data, final performance should be measured on independent test data or by a properly nested resampling design.[2]

### Validation

Random train-test splits can be noisy, especially when events are scarce. Cross-validation or bootstrap procedures can estimate internal performance more efficiently when the entire modeling pipeline is repeated within each resample. External validation tests the fixed modeling procedure or fixed fitted model in a distinct population, location, or time period.[8][19]

Reports should distinguish:

| Quantity | Example |
|---|---|
| Apparent performance | Performance on the data used to fit the model |
| Internal validation | Resampling-based estimate within the development source |
| Temporal validation | Later observations from the same setting |
| Geographic or institutional validation | Data from a different site |
| Prospective evaluation | Performance observed after the model and workflow are fixed |

Good discrimination in one setting does not imply stable calibration elsewhere. Recalibrating an intercept or slope may help under some shifts, but a changed predictor-outcome relationship can require model revision and new validation.[21]

## Extensions and related models

Binary logistic regression has several close relatives:

| Model | Response structure | Distinguishing feature |
|---|---|---|
| [Multi-class logistic regression](https://aiwiki.ai/wiki/multi-class_logistic_regression) | More than two unordered classes | Uses a multinomial likelihood or combines binary comparisons |
| Ordinal logistic regression | Ordered categories | Models cumulative or adjacent-category logits under additional constraints |
| Conditional logistic regression | Matched or stratified binary data | Conditions out stratum-specific nuisance intercepts |
| Mixed-effects logistic regression | Clustered binary data | Adds random effects for modeled between-cluster variation |
| Generalized estimating equations with logit link | Correlated binary data | Targets population-average parameters with a working correlation structure |
| Bayesian logistic regression | Binary data with prior distributions | Produces a posterior distribution over coefficients and predictions |

The multinomial and ordinal models have different likelihoods and coefficient interpretations, so they should not be treated as simple synonyms for the binary model.[3][8]

### Relation to linear regression and neural networks

Unlike [linear regression](https://aiwiki.ai/wiki/linear_regression), logistic regression models a Bernoulli mean through a link function and is normally fitted with a binomial likelihood. Ordinary least squares can be applied mechanically to a 0/1 response, producing a linear probability model, but that is a different model with different variance and range properties. It is inaccurate to say that squared error becomes unusable merely because a sigmoid appears; squared-error training of a sigmoid model is possible, although the Bernoulli log-likelihood is the standard objective and has cleaner statistical and optimization properties.[1][3]

A single [neural network](https://aiwiki.ai/wiki/neural_network) output unit with a sigmoid activation and binary cross-entropy can have the same input-output form as binary logistic regression when the unit receives a fixed feature vector through one affine transformation. A deep network is not simply a stack of independent logistic regressions. Its hidden layers learn nonlinear representations jointly, and many modern hidden units do not use sigmoid activations. The mathematical overlap is real but bounded.[3]

## Software

Different packages use the same model name with different defaults. Reproducible work should record package versions, preprocessing, solver, regularization, convergence tolerance, class weighting, and random seeds where relevant.

### Scikit-learn

As of scikit-learn 1.9.0, released before the 28 July 2026 research cutoff, `sklearn.linear_model.LogisticRegression` applies regularization by default and uses `lbfgs` as its default solver. `C` is the inverse regularization strength, and smaller finite values mean stronger shrinkage. Version 1.9 documents `penalty` as deprecated in favor of `l1_ratio` and `C`; `C=np.inf` requests no regularization. Solver support differs across L1, L2, elastic-net, and multinomial losses.[12][25]

```python
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

model = make_pipeline(
    StandardScaler(),
    LogisticRegression(
        C=1.0,
        solver="lbfgs",
        max_iter=1000,
    ),
)
model.fit(X_train, y_train)
probability = model.predict_proba(X_test)[:, 1]
```

The scaler in this example must be fitted within the training pipeline. It is not required for every solver or every feature type, and sparse matrices may require a scaler that does not center the data. The code returns probabilities; a final application still needs an evaluation plan and, if labels are required, a justified [decision threshold](https://aiwiki.ai/wiki/decision_threshold).

### R

Base R fits an ordinary binomial generalized linear model with `glm`:

```r
fit <- glm(
  outcome ~ age + treatment + age:treatment,
  family = binomial(link = "logit"),
  data = training_data
)
predicted_probability <- predict(
  fit,
  newdata = test_data,
  type = "response"
)
```

R's `binomial` family accepts several response encodings, including a factor, a 0/1 vector, or a two-column success/failure matrix. The official documentation describes `glm` fitting through iteratively reweighted least squares and identifies `logit` as the binomial default link.[26][27]

### Statsmodels

Statsmodels 0.14.6 provides both a discrete `Logit` class and a binomial `GLM`. `Logit` does not add an intercept automatically, so a constant must be included explicitly when wanted.[28]

```python
import statsmodels.api as sm

X_with_intercept = sm.add_constant(X_train)
fit = sm.Logit(y_train, X_with_intercept).fit()
predicted_probability = fit.predict(sm.add_constant(X_test))
```

This interface is oriented toward likelihood results and inference. Its defaults should not be assumed to match scikit-learn's regularized classifier.

## Reporting checklist

A logistic regression result is easier to audit when the report states:

| Item | Minimum useful detail |
|---|---|
| Outcome | Event definition, coding, observation window, and prevalence |
| Population | Inclusion, exclusion, sampling, clustering, and data dates |
| Predictors | Units, encodings, reference levels, interactions, and nonlinear terms |
| Missing data | Amount, assumed mechanism, and handling within resampling |
| Estimation | Package version, solver, penalty, weights, and convergence result |
| Separation | Whether it was checked and how any problem was handled |
| Target estimand | Conditional odds ratio, marginal contrast, probability prediction, ranking, or decision |
| Internal validation | Resampling design with the full pipeline repeated |
| Performance | Calibration, discrimination, probability loss, and uncertainty |
| Threshold | Selection rule, validation separation, and consequences of errors |
| External validity | Population, time, or site differences from development data |

This information matters more than presenting a coefficient table alone. A fitted equation can be mathematically correct while answering the wrong scientific question or failing in its intended population.

## Strengths and limitations

| Strength | Corresponding limit |
|---|---|
| Produces bounded conditional probabilities | Probabilities can still be miscalibrated under misspecification or distribution shift |
| Coefficients have a conditional log-odds interpretation | Odds ratios are not risk ratios and are noncollapsible |
| Unpenalized negative log-likelihood is convex | Separation can prevent a finite maximum-likelihood estimate |
| Supports sparse and dense design matrices | High dimensionality requires regularization and careful validation |
| Accommodates interactions and nonlinear basis terms | These terms must be specified; they are not learned automatically in the basic model |
| Fast implementations scale to large problems | Solver defaults and penalties differ across software |
| Useful for inference and prediction | The valid workflow and interpretation differ between those goals |

Logistic regression remains useful because its assumptions and outputs can be stated precisely. Its apparent simplicity is also a common source of error: a 0.5 cutoff, ten events per variable, automatic calibration, or causal interpretation cannot be treated as universal properties of the model.

## See also

- [Classification](https://aiwiki.ai/wiki/classification)
- [Generalized linear model](https://aiwiki.ai/wiki/generalized_linear_model)
- [Multi-class logistic regression](https://aiwiki.ai/wiki/multi-class_logistic_regression)
- [Regularization](https://aiwiki.ai/wiki/regularization)
- [Scikit-learn](https://aiwiki.ai/wiki/scikit_learn)
- [Interpretability](https://aiwiki.ai/wiki/interpretability)

## References

[1] James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. An Introduction to Statistical Learning: With Applications in R. 2nd ed., corrected printing, 21 June 2023. [Official book site](https://www.statlearning.com/).
[2] Scikit-learn developers. "Tuning the decision threshold for class prediction." Scikit-learn 1.9.0 documentation, 2026. [Official documentation](https://scikit-learn.org/1.9/modules/classification_threshold.html).
[3] Hastie, Trevor, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed., 12th printing, Springer, 2017. [Official author page](https://hastie.su.domains/ElemStatLearn/).
[4] Cramer, J. S. "The Origins of Logistic Regression." Tinbergen Institute Discussion Paper 2002-119/4, December 2002. [University of Amsterdam repository](https://dare.uva.nl/id/6d9e8c71-0ba0-458d-bd84-a6461cefd2ce).
[5] Berkson, Joseph. "Application of the Logistic Function to Bio-Assay." Journal of the American Statistical Association, vol. 39, no. 227, 1944, pp. 357-365. [DOI](https://doi.org/10.1080/01621459.1944.10500699).
[6] Cox, D. R. "The Regression Analysis of Binary Sequences." Journal of the Royal Statistical Society: Series B, vol. 20, no. 2, 1958, pp. 215-232. [DOI](https://doi.org/10.1111/j.2517-6161.1958.tb00292.x).
[7] Nelder, J. A., and R. W. M. Wedderburn. "Generalized Linear Models." Journal of the Royal Statistical Society: Series A, vol. 135, no. 3, 1972, pp. 370-384. [DOI](https://doi.org/10.2307/2344614).
[8] Harrell, Frank E. Regression Modeling Strategies. 2nd ed., Springer, 2015. [Publisher page](https://link.springer.com/book/10.1007/978-3-319-19425-7).
[9] Albert, A., and J. A. Anderson. "On the Existence of Maximum Likelihood Estimates in Logistic Regression Models." Biometrika, vol. 71, no. 1, 1984, pp. 1-10. [DOI](https://doi.org/10.1093/biomet/71.1.1).
[10] Friedman, Jerome, Trevor Hastie, and Robert Tibshirani. "Regularization Paths for Generalized Linear Models via Coordinate Descent." Journal of Statistical Software, vol. 33, no. 1, 2010, pp. 1-22. [DOI](https://doi.org/10.18637/jss.v033.i01).
[11] Fan, Rong-En, et al. "LIBLINEAR: A Library for Large Linear Classification." Journal of Machine Learning Research, vol. 9, 2008, pp. 1871-1874. [Paper](https://www.jmlr.org/papers/v9/fan08a.html).
[12] Scikit-learn developers. "sklearn.linear_model.LogisticRegression." Scikit-learn 1.9.0 documentation, 2026. [Official API reference](https://scikit-learn.org/1.9/modules/generated/sklearn.linear_model.LogisticRegression.html).
[13] Norton, Edward C., Bryan E. Dowd, and Matthew L. Maciejewski. "Marginal Effects: Quantifying the Effect of Changes in Risk Factors in Logistic Regression Models." JAMA, vol. 321, no. 13, 2019, pp. 1304-1305. [DOI](https://doi.org/10.1001/jama.2019.1954).
[14] Schuster, Noah A., et al. "Noncollapsibility and Its Role in Quantifying Confounding Bias in Logistic Regression." BMC Medical Research Methodology, vol. 21, article 136, 2021. [DOI](https://doi.org/10.1186/s12874-021-01316-8).
[15] Ng, Andrew Y. "Feature Selection, L1 vs. L2 Regularization, and Rotational Invariance." Proceedings of the Twenty-First International Conference on Machine Learning, 2004. [Author-hosted paper](https://ai.stanford.edu/~ang/papers/icml04-l1l2.pdf).
[16] Firth, David. "Bias Reduction of Maximum Likelihood Estimates." Biometrika, vol. 80, no. 1, 1993, pp. 27-38. [DOI](https://doi.org/10.1093/biomet/80.1.27).
[17] Peduzzi, Peter, et al. "A Simulation Study of the Number of Events per Variable in Logistic Regression Analysis." Journal of Clinical Epidemiology, vol. 49, no. 12, 1996, pp. 1373-1379. [PubMed](https://pubmed.ncbi.nlm.nih.gov/8970487/).
[18] van Smeden, Maarten, et al. "No Rationale for 1 Variable per 10 Events Criterion for Binary Logistic Regression Analysis." BMC Medical Research Methodology, vol. 16, article 163, 2016. [DOI](https://doi.org/10.1186/s12874-016-0267-3).
[19] Riley, Richard D., et al. "Calculating the Sample Size Required for Developing a Clinical Prediction Model." BMJ, vol. 368, 2020, m441. [DOI](https://doi.org/10.1136/bmj.m441).
[20] King, Gary, and Langche Zeng. "Logistic Regression in Rare Events Data." Political Analysis, vol. 9, no. 2, 2001, pp. 137-163. [Harvard repository](https://dash.harvard.edu/entities/publication/73120378-8901-6bd4-e053-0100007fdf3b).
[21] Van Calster, Ben, et al. "Calibration: The Achilles Heel of Predictive Analytics." BMC Medicine, vol. 17, article 230, 2019. [DOI](https://doi.org/10.1186/s12916-019-1466-7).
[22] Scikit-learn developers. "Probability calibration." Scikit-learn 1.9.0 documentation, 2026. [Official documentation](https://scikit-learn.org/1.9/modules/calibration.html).
[23] Fawcett, Tom. "An Introduction to ROC Analysis." Pattern Recognition Letters, vol. 27, no. 8, 2006, pp. 861-874. [DOI](https://doi.org/10.1016/j.patrec.2005.10.010).
[24] Saito, Takaya, and Marc Rehmsmeier. "The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets." PLOS ONE, vol. 10, no. 3, 2015, e0118432. [DOI](https://doi.org/10.1371/journal.pone.0118432).
[25] Scikit-learn developers. "Version 1.9.0." Scikit-learn release history, June 2026. [Official release notes](https://scikit-learn.org/1.9/whats_new/v1.9.html).
[26] R Core Team. "glm: Fitting Generalized Linear Models." R 4.6.1 documentation, released 24 June 2026. [Official manual](https://stat.ethz.ch/R-manual/R-patched/library/stats/html/glm.html).
[27] R Core Team. "family: Family Objects for Models." R 4.6.1 documentation, released 24 June 2026. [Official manual](https://stat.ethz.ch/R-manual/R-patched/library/stats/html/family.html).
[28] Statsmodels developers. "statsmodels.discrete.discrete_model.Logit." Statsmodels 0.14.6 documentation, updated 5 December 2025. [Official API reference](https://www.statsmodels.org/stable/generated/statsmodels.discrete.discrete_model.Logit.html).