# Exponential family

> Source: https://aiwiki.ai/wiki/exponential_family
> Updated: 2026-11-18
> Fact-checked: 2026-11-18
> Categories: Machine Learning, Mathematics, Statistics
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Exponential family." aiwiki.ai, 18 Nov 2026. https://aiwiki.ai/wiki/exponential_family
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

An **exponential family** is a class of [probability distributions](https://aiwiki.ai/wiki/probability_distribution) whose density or mass function can be written as the exponential of a linear function of a fixed set of statistics of the data, multiplied by a base function of the data and divided by a normalizing constant that depends only on the parameters. The class includes the Bernoulli, Poisson, Gaussian, exponential, beta, gamma, Dirichlet and multinomial distributions, among many others [1][2][3]. The shared form gives every member the same toolkit: a sufficient statistic of fixed size, moments obtained by differentiating one convex function, maximum likelihood estimates that match observed and expected statistics, and conjugate priors. That structure is why exponential families sit underneath [generalized linear models](https://aiwiki.ai/wiki/generalized_linear_model), [logistic regression](https://aiwiki.ai/wiki/logistic_regression), the [expectation-maximization](https://aiwiki.ai/wiki/expectation_maximization) algorithm, many [probabilistic graphical models](https://aiwiki.ai/wiki/probabilistic_graphical_model), and much of [variational inference](https://aiwiki.ai/wiki/variational_inference) [1][3].

The family was characterized in the mid-1930s by Georges Darmois, Bernard Koopman and E. J. G. Pitman, who showed that it is tied to the existence of sufficient statistics that do not grow with the sample size [4][5][7]. Bradley Efron's 2022 textbook describes exponential families as having reached a position at the center of parametric statistical inference over the preceding half-century [19].

## Definition

A parametric family of distributions for a random variable $X$ is an exponential family if its densities, taken with respect to some base measure (Lebesgue measure for continuous data, counting measure for discrete data), can be written as [1][2]

$$
p(x \mid \theta) = h(x)\,\exp\big(\langle \theta, T(x)\rangle - A(\theta)\big)
$$

The parts have standard names:

| Symbol | Name | Role |
|---|---|---|
| $T(x)$ | Sufficient statistic (also called potential functions) | A vector of functions of the data |
| $\theta$ | Natural or canonical parameter | A vector paired with $T(x)$ by an inner product |
| $h(x)$ | Base measure or carrier | A function of the data alone |
| $A(\theta)$ | Log partition function or cumulant function | The log of the normalizing constant |

The log partition function is fixed once $T$, $h$ and the base measure are chosen, because it must make the density integrate to one [2]:

$$
A(\theta) = \log \int h(x)\, \exp\big(\langle \theta, T(x)\rangle\big)\, \nu(dx)
$$

The natural parameter space is the set of $\theta$ for which this integral is finite. Wainwright and Jordan's survey calls a family **regular** when that set is open, and **minimal** when no nonzero linear combination of the sufficient statistics is constant almost everywhere, so that each distribution corresponds to exactly one parameter vector [1]. A representation that is not minimal is called **overcomplete**; it loses identifiability of $\theta$ but can be convenient, for example when analyzing message-passing algorithms [1]. Michael Jordan's lecture notes add the notion of a **curved exponential family**, where the parameters are restricted to a nonlinear subset of the natural parameter space that cannot be reduced to a smaller exponential family [2].

Many familiar distributions are written with a different parameter, such as the success probability of a Bernoulli variable or the rate of a Poisson variable. Rewriting the density as the exponential of its logarithm and collecting terms that multiply functions of $x$ exposes the natural parameter [2]. For regression models, a slightly more general form adds a dispersion parameter that divides the exponent; for the Gaussian, the dispersion is the variance $\sigma^2$ [3].

<figure class="embedvideo">
<iframe src="https://www.youtube-nocookie.com/embed/xwM9XcnQ4Us" title="The Exponential Family (Part 1)"></iframe>
<figcaption>"The Exponential Family (Part 1)", video from Mutual Information, published July 28, 2021. <a href="https://www.youtube.com/watch?v=xwM9XcnQ4Us">Watch on YouTube</a>.</figcaption>
</figure>

## Examples

The following table lists common members in natural form. Entries follow Wainwright and Jordan's Table 3.1 and Jordan's worked examples [1][2].

| Distribution | Support | $T(x)$ | Natural parameter | $A(\theta)$ |
|---|---|---|---|---|
| Bernoulli, $P(x{=}1) = \pi$ | $\{0, 1\}$ | $x$ | $\eta = \log\frac{\pi}{1-\pi}$ | $\log(1 + e^{\eta})$ |
| Poisson, rate $\lambda$ | $\{0, 1, 2, \dots\}$ | $x$ | $\eta = \log \lambda$ | $e^{\eta}$ |
| Gaussian, mean $\mu$, variance $\sigma^2$ | $\mathbb{R}$ | $(x, x^2)$ | $\eta = (\mu/\sigma^2,\ -1/(2\sigma^2))$ | $-\frac{\eta_1^2}{4\eta_2} - \frac{1}{2}\log(-2\eta_2)$ |
| Exponential | $(0, \infty)$ | $x$ | $\theta < 0$ | $-\log(-\theta)$ |
| Beta | $(0, 1)$ | $(\log x, \log(1-x))$ | $\theta \in (-1, \infty)^2$ | $\sum_i \log\Gamma(\theta_i + 1) - \log\Gamma\big(\sum_i (\theta_i + 1)\big)$ |
| Categorical ($K$ outcomes) | one of $K$ categories | indicator vector | $\eta_k = \log(\pi_k / \pi_K)$ | $\log \sum_k e^{\eta_k}$ |

Inverting the natural parameter recovers familiar link functions. For the Bernoulli distribution, solving $\eta = \log(\pi/(1-\pi))$ for $\pi$ gives the logistic [sigmoid function](https://aiwiki.ai/wiki/sigmoid_function) $\pi = 1/(1 + e^{-\eta})$ [2][3]. For the categorical distribution, the same inversion gives $\pi_k = e^{\eta_k} / \sum_j e^{\eta_j}$, which Jordan's notes identify as the multinomial logit or [softmax](https://aiwiki.ai/wiki/softmax) function [2]. The Gaussian needs a two-dimensional sufficient statistic when both the mean and the variance are unknown; with the variance fixed at 1, as in Wainwright and Jordan's Gaussian location family and the CS229 notes, it reduces to a one-parameter family whose natural parameter is the mean itself [1][3]. Jordan's notes also work through the von Mises distribution for angles as a member [2], and the Stanford CS229 notes list the multinomial, Poisson, gamma, exponential, beta and Dirichlet distributions as further examples [3].

The link between exponential families and sufficient statistics of fixed size needs care when the range of the data depends on the parameter. Pitman's 1936 paper already noted that his theory of the intrinsic accuracy of statistics needs modification when the range of the population depends on the parameter [7], and Robert, Chopin and Rousseau use the uniform distribution to show why the classical theorem needs a support that does not depend on the parameter [4].

## History

| Year | Contribution | Source |
|---|---|---|
| 1935 | Georges Darmois publishes a note in *Comptes Rendus* on distributions admitting a sufficient statistic | [4][5] |
| 1936 | Bernard Koopman, "On distributions admitting a sufficient statistic" | [6] |
| 1936 | E. J. G. Pitman, "Sufficient statistics and intrinsic accuracy" | [7] |
| 1957 | E. T. Jaynes connects information theory and statistical mechanics; maximum entropy later used to motivate exponential families | [1][8] |
| 1972 | Nelder and Wedderburn introduce generalized linear models for exponential-family responses | [9] |
| 1975 | Efron defines statistical curvature, which is zero for one-parameter exponential families and positive otherwise | [11] |
| 1979 | Diaconis and Ylvisaker characterize conjugate priors for exponential families | [10] |
| 1982 | Morris classifies natural exponential families with quadratic variance functions | [12] |
| 2001-2005 | Machine learning generalizations: exponential family PCA, exponential family harmoniums, Bregman clustering | [13][14][15] |
| 2008 | Wainwright and Jordan's survey of graphical models, exponential families and variational inference | [1] |
| 2013 | Stochastic variational inference for conditionally conjugate exponential-family models | [16] |

The name and the core theorem come from three nearly simultaneous papers. Darmois published a version of the result in French in 1935, about a year before Pitman and Koopman published theirs in 1936 [4]. Pitman's paper proved that if a sufficient statistic exists for an unknown parameter, the population's frequency function must be of a particular type, and it extended the analysis to sets of jointly sufficient statistics [7]. The MacTutor biography of Darmois, quoting a 2010 paper by John Aldrich, notes that the distributions Darmois studied are now called the exponential, or Koopman-Pitman-Darmois, family [5].

Book-length treatments followed, including Ole Barndorff-Nielsen's *Information and Exponential Families* (Wiley, 1978) and Lawrence Brown's *Fundamentals of Statistical Exponential Families* (Institute of Mathematical Statistics, 1986), both cited by Wainwright and Jordan for the statistical theory of these families [1]. Later books include Rolf Sundberg's *Statistical Modelling by Exponential Families* (Cambridge University Press, 2019) and Efron's *Exponential Families in Theory and Practice* (Cambridge University Press, 2022) [18][19].

## Properties

### Moments from the log partition function

The log partition function generates the moments of the sufficient statistic. For a regular family, $A$ has derivatives of all orders on the interior of its domain, and [1][2]

$$
\nabla A(\theta) = \mathbb{E}_\theta[T(X)], \qquad \nabla^2 A(\theta) = \operatorname{Cov}_\theta[T(X)].
$$

Because the Hessian is a covariance matrix, it is positive semidefinite, so $A$ is a [convex function](https://aiwiki.ai/wiki/convex_function), strictly convex when the representation is minimal, and the natural parameter space is a convex set [1][2]. For the Bernoulli case, differentiating $A(\eta) = \log(1 + e^\eta)$ once gives the mean $\mu = 1/(1+e^{-\eta})$, and differentiating again gives the variance $\mu(1-\mu)$ [2]. Higher cumulants come from higher derivatives: the cumulant generating function of $T(X)$ is $C(s) = A(s + \theta) - A(\theta)$, which is why $A$ is called the cumulant function [2].

### Mean parameterization

The vector $\mu = \mathbb{E}_\theta[T(X)] = \nabla A(\theta)$ is called the mean parameter. In a minimal family, the gradient map $\theta \mapsto \nabla A(\theta)$ is one-to-one, and it maps the natural parameter space onto the interior of the set of all mean vectors that any distribution could produce [1]. Every distribution in a minimal family can therefore be indexed either by its natural parameters or by its mean parameters [1][2]. The inverse map is the gradient of the convex conjugate $A^*(\mu) = \sup_\theta \{\langle \mu, \theta\rangle - A(\theta)\}$, and for mean parameters in the interior, $A^*(\mu)$ equals the negative [entropy](https://aiwiki.ai/wiki/entropy) of the matching exponential-family distribution [1]. This duality gives the variational representation

$$
A(\theta) = \sup_{\mu \in \mathcal{M}} \big\{\langle \theta, \mu\rangle - A^*(\mu)\big\},
$$

where $\mathcal{M}$ is the set of realizable mean parameters. Wainwright and Jordan build their account of exact and approximate inference algorithms on this identity [1].

### Sufficiency

The factorization of the density shows directly that $T(X)$ is a sufficient statistic for $\theta$ [2]. For $N$ [independent and identically distributed](https://aiwiki.ai/wiki/independently_and_identically_distributed_i_i_d) observations, the joint density is again an exponential family with sufficient statistic $\sum_{n=1}^N T(x_n)$. For Bernoulli, Poisson and categorical data this is simply the sum of the observations; for a Gaussian with unknown mean and variance it is the pair (sum of observations, sum of squares). All other details of the sample can be discarded without losing information about the parameter [2].

The Pitman-Koopman-Darmois theorem gives the converse direction: under regularity conditions that include a support that does not depend on the parameter, only exponential families admit sufficient statistics whose dimension stays fixed as the sample grows [4][7]. Efron's 1975 paper made a related point geometrically, describing one-parameter exponential families as "straight lines" through the space of distributions and defining a statistical curvature that is zero for exponential families and positive for other families [11].

### Maximum likelihood

With i.i.d. data, the average log-likelihood is $\langle \theta, \hat\mu\rangle - A(\theta)$ (plus a term that does not depend on $\theta$), where $\hat\mu = \frac{1}{N}\sum_n T(x_n)$ is the vector of empirical averages of the sufficient statistics [1][2]. Setting the gradient to zero gives the moment-matching condition

$$
\nabla A(\hat\theta) = \mathbb{E}_{\hat\theta}[T(X)] = \frac{1}{N}\sum_{n=1}^N T(x_n).
$$

The [maximum likelihood estimate](https://aiwiki.ai/wiki/maximum_likelihood_estimation) of the mean parameter is thus the sample average of the sufficient statistic, and in the univariate case it attains the Cramer-Rao lower bound [2]. The [Fisher information](https://aiwiki.ai/wiki/fisher_information) for the natural parameter equals $\nabla^2 A(\theta)$, the covariance of the sufficient statistic [2][16]. Finding $\hat\theta$ from $\hat\mu$ is the "backward" mapping from mean to natural parameters; it is easy for simple families but can be computationally hard for high-dimensional ones such as large graphical models [1].

### Maximum entropy

Exponential families also arise as solutions to a maximum entropy problem. Suppose the expectations of some functions $T(x)$ are known. Among all distributions that match those expectations, the one with the largest Shannon entropy has the exponential-family form, with the natural parameters playing the role of Lagrange multipliers for the moment constraints [1]. Wainwright and Jordan show that this problem and maximum likelihood are dual to each other: both lead to the same moment-matching conditions [1]. They cite Jaynes's 1957 paper as a source of the maximum entropy principle [1][8].

### KL divergence and Bregman divergences

The [Kullback-Leibler divergence](https://aiwiki.ai/wiki/kl_divergence) between two members of the same family has a closed form in terms of the log partition function [1][2]:

$$
D(\theta_1 \,\|\, \theta_2) = A(\theta_2) - A(\theta_1) - \langle \mu_1, \theta_2 - \theta_1\rangle,
$$

where $\mu_1 = \nabla A(\theta_1)$. Geometrically, this is the gap between $A(\theta_2)$ and the tangent plane to $A$ at $\theta_1$, which makes the KL divergence a Bregman divergence generated by $A$ [1]. Banerjee, Merugu, Dhillon and Ghosh proved in 2005 that there is a bijection between regular exponential families and a large class of Bregman divergences, which they called regular Bregman divergences [15].

### Conjugate priors

Every exponential-family likelihood has a conjugate prior that is itself an exponential family. If the likelihood is $p(x \mid \beta) = h(x)\exp\{\beta^\top t(x) - a(\beta)\}$, a prior with sufficient statistics $(\beta, -a(\beta))$ and hyperparameters $\alpha = (\alpha_1, \alpha_2)$ yields a posterior in the same family, with natural parameter [16]

$$
\Big(\alpha_1 + \sum_{n=1}^N t(x_n),\ \ \alpha_2 + N\Big).
$$

Updating the posterior therefore reduces to adding the observed sufficient statistics and the sample size to the hyperparameters. Diaconis and Ylvisaker's 1979 paper characterized these conjugate priors by the property that the posterior expectation of the mean parameter is a linear function of the observation, and determined which hyperparameters make the priors proper [10]. Conjugate exponential-family pairs are the building blocks of the conditionally conjugate models that classical and stochastic variational [Bayesian inference](https://aiwiki.ai/wiki/bayesian_inference) algorithms handle in closed form [16].

### Variance functions

Carl Morris studied natural exponential families through their variance functions, which express the variance as a function of the mean. He showed in 1982 that the normal, Poisson, gamma, binomial and negative binomial distributions are natural exponential families whose variance is at most a quadratic function of the mean, and that exactly one other such family exists [12]. His follow-up paper used this shared property to unify results on unbiased estimation, conjugate priors and empirical Bayes methods for all six families [12].

## Role in generalized linear models

Generalized linear models (GLMs), introduced by John Nelder and Robert Wedderburn in 1972, are regression models in which the response follows an exponential-family distribution and the systematic effects can be made linear by a transformation. Their paper showed that maximum likelihood estimates for such models can be found by iterative weighted linear regression, and illustrated the method with the normal, binomial, Poisson and gamma distributions [9].

The Stanford CS229 notes by Tengyu Ma and Andrew Ng present the construction as three assumptions [3]:

1. Given the inputs $x$ and parameters, the response $y$ follows an exponential-family distribution with natural parameter $\eta$.
2. The model predicts the expected value of $T(y)$, which is usually $\mathbb{E}[y \mid x]$.
3. The natural parameter is linear in the inputs: $\eta = \theta^\top x$.

The function that gives the distribution's mean as a function of the natural parameter is called the canonical response function, and its inverse is the canonical link function [3]. With a Gaussian response, the canonical response is the identity and the GLM is ordinary least squares [linear regression](https://aiwiki.ai/wiki/linear_regression). With a Bernoulli response, the canonical response is the logistic function, and the GLM is logistic regression; in the notes' words, the logistic form "arises as a consequence of the definition of GLMs and exponential family distributions" [3]. A Poisson response gives a model for counts such as the number of customers arriving per hour [3]. The categorical case, whose mean map is the softmax function, underlies multiclass logistic regression and the softmax output layers of classification networks [2][3].

## Role in variational inference and graphical models

### Graphical models as exponential families

Many [probabilistic graphical models](https://aiwiki.ai/wiki/probabilistic_graphical_model) are exponential families whose sufficient statistics are attached to the nodes and cliques of a graph. Wainwright and Jordan's 2008 survey works through the Ising model from statistical physics, the Potts model, Gaussian Markov random fields, finite mixture models such as [Gaussian mixture models](https://aiwiki.ai/wiki/gaussian_mixture_model), and [latent Dirichlet allocation](https://aiwiki.ai/wiki/latent_dirichlet_allocation) in this form [1]. Products of local factors in the usual graphical-model notation become sums inside the exponent [1].

In this view, two central computational problems are mappings between the two parameterizations. Computing marginal probabilities, the main inference task, amounts to computing the forward map from natural parameters $\theta$ to mean parameters $\mu$; maximum likelihood estimation amounts to the backward map from empirical means to $\theta$. Both mappings are easy for low-dimensional families but can be very hard for high-dimensional graphical models, which is what motivates approximate methods [1].

### Mean field and the variational principle

The variational representation $A(\theta) = \sup_{\mu} \{\langle \theta, \mu\rangle - A^*(\mu)\}$ is exact but usually intractable, because the set of realizable mean parameters can be complicated and $A^*$ often has no explicit form [1]. Mean field methods restrict the optimization to mean parameters produced by a tractable subfamily, such as fully factorized (product) distributions or tree-structured distributions. Any valid mean parameter gives a lower bound on $A(\theta)$, and finding the best bound within the tractable subfamily is equivalent to minimizing the KL divergence $D(q \,\|\, p_\theta)$ between a tractable approximation $q$ and the target $p_\theta$ [1].

### Expectation-maximization

When a model is an exponential family over observed and hidden variables jointly, the E step of the [EM algorithm](https://aiwiki.ai/wiki/expectation_maximization) reduces to computing expected sufficient statistics, which are mean parameters of the conditional distribution of the hidden variables, and the M step is an ordinary maximum likelihood problem using those expected statistics [1]. Seen this way, EM is coordinate ascent on a lower bound of the log-likelihood. Replacing the exact E step with a variational approximation gives variational EM; with mean field approximations, the method still performs coordinate ascent on a lower bound [1].

### Stochastic variational inference and natural gradients

Matthew Hoffman, David Blei, Chong Wang and John Paisley's 2013 stochastic variational inference (SVI) paper assumes that every complete conditional distribution in the model is an exponential family and that the global parameters have a conjugate exponential-family prior [16]. The authors list Bayesian mixture models, latent Dirichlet allocation, [hidden Markov models](https://aiwiki.ai/wiki/hidden_markov_model) and [Kalman filters](https://aiwiki.ai/wiki/kalman_filter) among the models in this class [16]. When the variational distribution is in the same exponential family, its Fisher information metric is the Hessian of the log partition function, and the natural gradient of the evidence lower bound takes the simple form of the expected natural parameter of the complete conditional minus the current variational parameter [16]. A natural gradient step of length one is equivalent to the classical coordinate ascent update, and following noisy but unbiased natural gradients computed from subsampled data gives the stochastic algorithm, which the authors applied to millions of documents with topic models [16].

Mohammad Emtiyaz Khan and Håvard Rue's 2023 "Bayesian learning rule" generalizes this approach. It optimizes over a regular, minimal exponential family of candidate posteriors using natural gradients, and it can be read as mirror descent with a Bregman divergence set by the chosen family [17]. Different candidate families and approximations recover different algorithms: conjugate Bayesian updating, SVI and variational message passing from exponential-family candidates, EM from an exponential family combined with a Gaussian, and optimizers such as [stochastic gradient descent](https://aiwiki.ai/wiki/stochastic_gradient_descent_sgd), [RMSprop](https://aiwiki.ai/wiki/rmsprop) and [Adam](https://aiwiki.ai/wiki/adam_optimizer) from Gaussian candidates [17].

## Other uses in machine learning

Several machine learning methods were built by replacing a Gaussian assumption with a general exponential family:

| Method | Authors, year | Idea |
|---|---|---|
| Exponential family PCA | Collins, Dasgupta, Schapire (2001) | [Principal component analysis](https://aiwiki.ai/wiki/principal_component_analysis) implicitly minimizes squared loss, which can be inappropriate for data that is not real-valued; the paper uses exponential families, GLMs and Bregman distances to derive losses suited to other data types [13] |
| Exponential family harmoniums | Welling, Rosen-Zvi, Hinton (2004) | Extends the harmonium, also studied as the restricted [Boltzmann machine](https://aiwiki.ai/wiki/boltzmann_machine), from binary and Gaussian units to exponential-family units, trained by contrastive divergence; applied as an alternative to latent semantic indexing [14] |
| Clustering with Bregman divergences | Banerjee, Merugu, Dhillon, Ghosh (2005) | Uses the exponential family to Bregman divergence bijection to generalize [k-means](https://aiwiki.ai/wiki/k-means) to a broad class of losses while keeping its simplicity and scalability [15] |

The common thread is replacing squared error, which suits real-valued data, with losses derived from other exponential families [13][15].

## See also

- [Generalized linear model](https://aiwiki.ai/wiki/generalized_linear_model)
- [Variational inference](https://aiwiki.ai/wiki/variational_inference)
- [Expectation-maximization](https://aiwiki.ai/wiki/expectation_maximization)
- [Maximum likelihood estimation](https://aiwiki.ai/wiki/maximum_likelihood_estimation)
- [KL divergence](https://aiwiki.ai/wiki/kl_divergence)
- [Probabilistic graphical model](https://aiwiki.ai/wiki/probabilistic_graphical_model)
- [Bayesian statistics](https://aiwiki.ai/wiki/bayesian_statistics)

## References

1. Wainwright, M. J.; Jordan, M. I. "Graphical Models, Exponential Families, and Variational Inference". Foundations and Trends in Machine Learning, vol. 1, no. 1-2, pp. 1-305, 2008. https://doi.org/10.1561/2200000001
2. Jordan, M. I. "The exponential family: Basics". Chapter 8 of an unpublished book draft, University of California, Berkeley. https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/other-readings/chapter8.pdf
3. Ma, T.; Ng, A. "CS229 Lecture Notes" (Chapter 3, Generalized linear models). Stanford University, 2026. https://cs229.stanford.edu/main_notes.pdf
4. Robert, C. P.; Chopin, N.; Rousseau, J. "Harold Jeffreys's Theory of Probability Revisited". Statistical Science, vol. 24, no. 2, pp. 141-172, 2009. https://arxiv.org/abs/0804.3173
5. O'Connor, J. J.; Robertson, E. F. "Georges Darmois". MacTutor History of Mathematics Archive, University of St Andrews, 2011. https://mathshistory.st-andrews.ac.uk/Biographies/Darmois/
6. Koopman, B. O. "On distributions admitting a sufficient statistic". Transactions of the American Mathematical Society, vol. 39, no. 3, pp. 399-409, 1936. https://doi.org/10.1090/s0002-9947-1936-1501854-3
7. Pitman, E. J. G. "Sufficient statistics and intrinsic accuracy". Mathematical Proceedings of the Cambridge Philosophical Society, vol. 32, no. 4, pp. 567-579, 1936. https://doi.org/10.1017/s0305004100019307
8. Jaynes, E. T. "Information Theory and Statistical Mechanics". Physical Review, vol. 106, no. 4, pp. 620-630, 1957. https://doi.org/10.1103/physrev.106.620
9. Nelder, J. A.; Wedderburn, R. W. M. "Generalized linear models". Journal of the Royal Statistical Society, Series A, vol. 135, no. 3, pp. 370-384, 1972. https://doi.org/10.2307/2344614
10. Diaconis, P.; Ylvisaker, D. "Conjugate Priors for Exponential Families". The Annals of Statistics, vol. 7, no. 2, pp. 269-281, 1979. https://doi.org/10.1214/aos/1176344611
11. Efron, B. "Defining the Curvature of a Statistical Problem (with Applications to Second Order Efficiency)". The Annals of Statistics, vol. 3, no. 6, pp. 1189-1242, 1975. https://doi.org/10.1214/aos/1176343282
12. Morris, C. N. "Natural Exponential Families with Quadratic Variance Functions". The Annals of Statistics, vol. 10, no. 1, pp. 65-80, 1982 (sequel: "Natural Exponential Families with Quadratic Variance Functions: Statistical Theory", vol. 11, no. 2, pp. 515-529, 1983, https://doi.org/10.1214/aos/1176346158). https://doi.org/10.1214/aos/1176345690
13. Collins, M.; Dasgupta, S.; Schapire, R. E. "A Generalization of Principal Components Analysis to the Exponential Family". Advances in Neural Information Processing Systems 14 (NIPS 2001), 2001. https://proceedings.neurips.cc/paper/2001/hash/f410588e48dc83f2822a880a68f78923-Abstract.html
14. Welling, M.; Rosen-Zvi, M.; Hinton, G. E. "Exponential Family Harmoniums with an Application to Information Retrieval". Advances in Neural Information Processing Systems 17 (NIPS 2004), 2004. https://papers.nips.cc/paper/2004/hash/0e900ad84f63618452210ab8baae0218-Abstract.html
15. Banerjee, A.; Merugu, S.; Dhillon, I. S.; Ghosh, J. "Clustering with Bregman Divergences". Journal of Machine Learning Research, vol. 6, pp. 1705-1749, 2005. https://jmlr.org/papers/v6/banerjee05b.html
16. Hoffman, M. D.; Blei, D. M.; Wang, C.; Paisley, J. "Stochastic Variational Inference". Journal of Machine Learning Research, vol. 14, pp. 1303-1347, 2013. https://jmlr.org/papers/v14/hoffman13a.html
17. Khan, M. E.; Rue, H. "The Bayesian Learning Rule". Journal of Machine Learning Research, vol. 24, no. 281, pp. 1-46, 2023. https://jmlr.org/papers/v24/22-0291.html
18. Sundberg, R. "Statistical Modelling by Exponential Families". Cambridge University Press, 2019. https://doi.org/10.1017/9781108604574
19. Efron, B. "Exponential Families in Theory and Practice". Cambridge University Press, 2022. https://doi.org/10.1017/9781108773157

