# Probabilistic programming

> Source: https://aiwiki.ai/wiki/probabilistic_programming
> Updated: 2026-11-18
> Fact-checked: 2026-11-18
> Categories: Machine Learning, Programming Languages, Statistics
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Probabilistic programming." aiwiki.ai, 18 Nov 2026. https://aiwiki.ai/wiki/probabilistic_programming
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**Probabilistic programming** is an approach to statistical modeling and [machine learning](https://aiwiki.ai/wiki/machine_learning) in which a model is written as a computer program that makes random choices, and a general-purpose inference engine then works out which values of those random choices are consistent with observed data. A probabilistic programming language (PPL) adds two things to an ordinary language: a way to draw values at random from distributions, and a way to condition on observations. The program therefore describes a joint distribution over unknowns and data, and the system's job is to compute the conditional, or posterior, distribution of the unknowns given the data [1]. The field sits where statistics, machine learning and programming-language research meet, and its main promise is that the modeler writes the model while the system supplies the inference algorithm [1][2].

Well-known systems include Stan, PyMC, Pyro and its JAX-based sibling NumPyro, Turing.jl, Gen, and the Lisp-based research languages descended from Church [3][2][5][15][6][16][4]. Their inference engines draw on [Markov chain Monte Carlo](https://aiwiki.ai/wiki/markov_chain_monte_carlo) (especially Hamiltonian Monte Carlo and the No-U-Turn Sampler), [variational inference](https://aiwiki.ai/wiki/variational_inference), sequential Monte Carlo and [importance sampling](https://aiwiki.ai/wiki/importance_sampling) [1].

## Core idea

In conventional programming, a program takes inputs and produces outputs. Statistical modeling runs in the other direction: it starts from observed data $y$, posits a generative model $p(x, y)$ for how hidden quantities $x$ produce that data, and asks for the posterior $p(x \mid y)$. Van de Meent, Paige, Yang and Wood describe probabilistic programming as doing Bayesian inference with the tools of computer science: a programming language denotes the model, and statistical inference algorithms compute the conditional distribution of the program's inputs that could have produced the observed output [1]. By [Bayes' theorem](https://aiwiki.ai/wiki/bayes_theorem),

$$p(x \mid y) = \frac{p(y \mid x)\, p(x)}{p(y)},$$

where $p(x)$ is the prior, $p(y \mid x)$ is the likelihood and $p(y)$ is the normalizing constant (the evidence). A probabilistic program specifies the prior and likelihood implicitly through its `sample` and `observe` statements, and inference has to characterize the posterior over all the random choices made while the program runs [1].

What separates a PPL from an ordinary language with a random number generator is conditioning. Almost every language can simulate random values; a PPL also has syntax for declaring which values are observed and an evaluator that implements conditioning on them [1]. Van de Meent and colleagues call languages without such constructs "stochastic simulation languages" [1]. They also distinguish PPLs from graphical-model libraries in which ordinary code builds a model data structure: in a PPL the model is the one implied by running the program itself [1].

### A minimal example

The first probabilistic program in the textbook by van de Meent and colleagues is a beta-Bernoulli coin model, written in a Lisp-like language similar to Anglican [1]:

```clojure
(let [prior (beta a b)
      x (sample prior)
      likelihood (bernoulli x)
      y 1]
  (observe likelihood y)
  x)
```

Run forward with the `observe` statement ignored, the program would just return a draw of the coin's bias `x` from the prior. The authors use it to explain the simplest kind of inference engine: run the program many times, and at each `observe` record a weight that is larger when the sampled `x` makes the observed outcome more likely. With a sensible choice of weight, the weighted samples form an asymptotically exact representation of the posterior. This is likelihood weighting, a form of importance sampling, and it requires running the program many times rather than once [1].

The same model in Stan declares data, parameters and a model block [3]:

```stan
data {
  int<lower=0> N;
  array[N] int<lower=0, upper=1> y;
}
parameters {
  real<lower=0, upper=1> theta;
}
model {
  theta ~ beta(1, 1);
  y ~ bernoulli(theta);
}
```

The 2017 Stan paper printed this program with the older postfix array syntax (`int<lower=0, upper=1> y[N];`); that syntax was removed in Stan 2.33 in favor of the `array[N]` form shown here [3][7].

<figure class="embedvideo">
<iframe src="https://www.youtube-nocookie.com/embed/-8QMqSWU76Q" title="&quot;An Overview of Probabilistic Programming&quot; by Vikash K. Mansinghka"></iframe>
<figcaption>"&quot;An Overview of Probabilistic Programming&quot; by Vikash K. Mansinghka", video from Strange Loop Conference, published September 28, 2015. <a href="https://www.youtube.com/watch?v=-8QMqSWU76Q">Watch on YouTube</a>.</figcaption>
</figure>

## History

### Statistical modeling languages

Statistics developed specialized languages for writing models and answering queries early on, among them BUGS and later Stan [1]. BUGS (Bayesian inference Using Gibbs Sampling) let users write a Bayesian model in a declarative language and fit it by Gibbs sampling. A 2009 retrospective by its developers, David Lunn, David Spiegelhalter, Andrew Thomas and Nicky Best, credited it with raising awareness of Bayesian modeling in academia and industry over a 20-year life span, and also offered a critical appraisal of its shortcomings [8]. JAGS (Plummer, 2003) followed the same declarative graphical-model approach [3].

Stan, developed by a team most of whose members were at Columbia University, released version 1.0 in 2012 [3][10]. It is named after Stanislaw Ulam, one of the developers of the Monte Carlo method in the 1940s [10]. The Stan team described two main departures from BUGS and JAGS: an imperative language with typed variables, local variables and conditional statements, and an MCMC engine based on Hamiltonian Monte Carlo rather than Gibbs sampling or Metropolis-Hastings [3].

### Universal languages from machine learning and cognitive science

Church, presented at the 2008 Conference on Uncertainty in Artificial Intelligence by Noah Goodman, Vikash Mansinghka, Daniel Roy, Keith Bonawitz and Joshua Tenenbaum of MIT, was a "universal" language for stochastic generative processes [4]. It was based on a pure subset of [Scheme](https://aiwiki.ai/wiki/scheme), treated evaluation as sampling and its `query` primitive as conditional sampling, and introduced a stochastic memoizer that made many nonparametric models simple to write [4]. Its authors showed that queries on any Church program could be answered with Monte Carlo techniques [4]. The Pyro authors later described Church as an early universal PPL able to represent any computable probability distribution [5].

The Pyro authors describe Anglican and WebPPL as lightweight successors to Church, embedded in the general-purpose languages [Clojure](https://aiwiki.ai/wiki/clojure) and [JavaScript](https://aiwiki.ai/wiki/javascript), and Venture as another universal language with its own syntax and virtual machine [5]. Anglican, introduced by Frank Wood, Jan-Willem van de Meent and Vikash Mansinghka at AISTATS 2014, used particle Markov chain Monte Carlo for inference in Turing-complete probabilistic programs, including programs with stochastic recursion [11]. Its later compiled version translates programs into Clojure through a continuation-passing-style transformation and runs on the Java Virtual Machine [11].

In March 2013 the US [DARPA](https://aiwiki.ai/wiki/darpa) began the Probabilistic Programming for Advancing Machine Learning (PPAML) program, which aimed to greatly increase the number of people able to build machine learning applications. It was scheduled for 46 months in three phases through 2017, and DARPA lists it as complete [12].

### Deep learning frameworks and probabilistic programming

Several later PPLs were built on deep learning frameworks, which gave them GPU support and automatic differentiation. Edward, by Dustin Tran, David Blei and colleagues at Columbia University, was a library for probabilistic modeling, inference and model criticism built on [TensorFlow](https://aiwiki.ai/wiki/tensorflow), with variational inference, Hamiltonian Monte Carlo and stochastic gradient Langevin dynamics among its algorithms [13]. In November 2017 Uber AI Labs released Pyro, a deep probabilistic programming language built on [PyTorch](https://aiwiki.ai/wiki/pytorch), as open source [14]. Turing, a Julia PPL with composable inference, was presented at AISTATS 2018 [6]. NumPyro, a lightweight library that gives Pyro's modeling interface a NumPy-style backend on [JAX](https://aiwiki.ai/wiki/jax), followed in 2019 [15]. Gen, from the MIT Probabilistic Computing Project, was presented at PLDI 2019 [16][17].

PyMC traces its history to the early 2000s. Version 2.0, released in January 2009, implemented Metropolis-Hastings sampling with distributions written in Fortran, and a 2016 rewrite (PyMC3) added gradient-based samplers including Hamiltonian Monte Carlo and NUTS on top of the Theano library [2]. After Theano's discontinuation was announced in 2017, PyMC moved to Theano's fork Aesara and then forked Aesara into PyTensor [2].

### Timeline

| Year | Event |
| --- | --- |
| 2008 | Church presented at UAI [4] |
| 2009 | PyMC 2.0 released (January) [2] |
| 2009 | BUGS retrospective marks a 20-year life span [8] |
| 2012 | Stan 1.0 released [10] |
| 2013 | DARPA PPAML program begins (March) [12] |
| 2014 | Anglican presented at AISTATS; NUTS published in JMLR [11][18] |
| 2016 | Edward paper on arXiv; PyMC3 rewrite adds NUTS [13][2] |
| 2017 | Stan paper in Journal of Statistical Software; ADVI paper in JMLR; Pyro open-sourced (November) [3][19][14] |
| 2018 | Turing presented at AISTATS [6] |
| 2019 | Pyro paper in JMLR; Gen at PLDI; NumPyro paper on arXiv [5][16][15] |
| 2022 | PyMC 4.0 (June) and 5.0 (December) [20] |
| 2023 | LLaMPPL proposes SMC steering of LLMs with probabilistic programs [21] |
| 2026 | PyMC 6.0 (May); Stan 2.40 (September) [20][22] |

## Language design

### Static and dynamic models

Van de Meent and colleagues highlight one distinction as especially important, and structure their book around it: whether every program in a language denotes a model with a finite, fixed set of random variables [1]. In a "first-order" language without unbounded recursion, a program can be compiled into a finite graphical model (a [Bayesian network](https://aiwiki.ai/wiki/bayesian_network) or factor graph) and inference can use graph-based algorithms. In a "higher-order" language with recursion and dynamic control flow, the number of random variables can depend on the values of other random variables, and no such compilation is possible in general; inference must instead work by running the program [1]. BUGS-style languages and Stan sit on the static side, while Church, Anglican, WebPPL and Pyro allow dynamic control flow [1][5].

### Standalone and embedded languages

Some PPLs are standalone languages with their own syntax and compiler. A Stan program is translated to C++ and compiled, and it imperatively defines a log probability function over the parameters given data and constants [3]. Its code is organized into named blocks: data, transformed data, parameters, transformed parameters, model and generated quantities [3]. Gelman, Lee and Guo noted in 2015 that, from the user's side, Stan's main limitation was that it did not allow inference for discrete parameters [10].

Others are embedded in a host language. Pyro programs are ordinary [Python](https://aiwiki.ai/wiki/python) functions with two extra primitives, `pyro.sample` and `pyro.param`, that can be used with any Python control flow, including recursion, loops and conditionals [5]. The PyMC developers contrast three implementation strategies: PyMC builds an explicit computational graph in PyTensor, Pyro implements inference with effect handlers, and Stan compiles the program into a log-density function in C++ [2].

### Design trade-offs

The Pyro paper frames PPL design around four goals that often conflict: expressivity (data-dependent control flow and latent variables whose existence depends on other variables), scalability (large data sets and high-dimensional models), flexibility (model-specific inference) and minimality (sharing syntax and semantics with existing languages) [5]. Its comparison table rates Stan as limited to static control flow but with automated inference, Church and Venture as fully expressive but not scalable, and Edward as scalable on CPU and GPU but limited to static control flow [5].

## Inference methods

PPL inference engines are general algorithms that must work for any program a user can write in the language [1]. The main families are:

| Method | How it works | Example systems |
| --- | --- | --- |
| Gibbs sampling | Updates variables one at a time from their conditional distributions | BUGS [8] |
| Metropolis-Hastings | Proposes changes to random choices and accepts or rejects them | Church, PyMC 2 [4][2] |
| Hamiltonian Monte Carlo, NUTS | Uses gradients of the log density to make long, informed moves | Stan, PyMC, Pyro, NumPyro, Turing [3][2][5][15][6] |
| Likelihood weighting, importance sampling | Runs the program forward and weights each run by the likelihood of the observations | General evaluators [1] |
| Sequential Monte Carlo, particle MCMC | Propagates and resamples a population of partial executions | Anglican, Turing, PyMC [11][6][2] |
| Variational inference | Fits an approximating distribution by optimization | Stan (ADVI), Pyro (SVI), PyMC [19][5][2] |
| Optimization | Finds a posterior mode or penalized maximum likelihood estimate | Stan (L-BFGS) [3] |

### Hamiltonian Monte Carlo and NUTS

Hamiltonian Monte Carlo uses first-order gradient information to avoid the random-walk behavior of simpler samplers, but its performance depends on two user-set parameters: a step size and a number of steps. Matthew Hoffman and Andrew Gelman's No-U-Turn Sampler (NUTS), published in the Journal of Machine Learning Research in 2014, removes the need to set the number of steps by building a set of candidate points recursively and stopping when the trajectory starts to double back. Together with an adaptive step-size scheme, it can run with no hand-tuning, which the authors presented as suiting BUGS-style automatic inference engines [18]. Stan uses NUTS for MCMC, and NUTS is the default sampler for continuous variables in PyMC [10][2]. Because these methods need gradients, PPLs that use them rely on [automatic differentiation](https://aiwiki.ai/wiki/automatic_differentiation): Stan through its C++ Stan Math Library for reverse-mode automatic differentiation, PyMC through PyTensor, Pyro through PyTorch and NumPyro through JAX [3][2][5][15].

### Variational inference

[Variational inference](https://aiwiki.ai/wiki/variational_inference) replaces sampling with optimization. With a family of distributions $q(x; \lambda)$ indexed by parameters $\lambda$, it maximizes the evidence lower bound (ELBO)

$$\mathcal{L}(\lambda) = \mathbb{E}_{q(x;\lambda)}\left[\log \frac{p(y, x)}{q(x; \lambda)}\right] = \log p(y) - D_{\mathrm{KL}}\big(q(x;\lambda) \,\|\, p(x \mid y)\big),$$

so that raising the bound pushes $q$ toward the posterior in [KL divergence](https://aiwiki.ai/wiki/kl_divergence) [1]. Automatic Differentiation Variational Inference (ADVI), by Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman and David Blei, derives a variational algorithm automatically from the model and data, needs no conjugacy assumptions, and was deployed as part of Stan [19]. Pyro's primary algorithm is gradient-based stochastic variational inference, which estimates the objective by Monte Carlo and scales to large data sets through mini-batches [5].

### Programmable and composable inference

Automatic inference does not work well for every model. The Pyro authors note that advanced models often need inference engines tailored to the application [5], and the Gen paper argues that existing systems lacked the flexibility and efficiency needed for harder models in fields such as [computer vision](https://aiwiki.ai/wiki/computer_vision) and [robotics](https://aiwiki.ai/wiki/robotics) [16]. Two responses are composable and programmable inference. Turing combines MCMC operations on subsets of a model's variables, for example a Hamiltonian Monte Carlo engine for some variables and a particle Gibbs engine for others [6]. Gen introduces a generative function interface for encapsulating models, several interoperable modeling languages with different flexibility and efficiency trade-offs, combinators for common conditional-independence patterns, and an inference library for writing custom algorithms at a high level [16]. In Pyro, the guide (the approximate posterior or proposal used by algorithms such as SVI) is itself written as a Pyro program, and the Poutine library of effect handlers lets developers change how programs execute without touching the language internals [5].

### Amortized inference

Standard algorithms solve one inference problem at a time. Amortized inference instead pays an up-front training cost to learn an inference network, usually a [neural network](https://aiwiki.ai/wiki/neural_network), that maps a data set directly to the parameters of an approximate posterior, so that inference on new data is fast [1]. Van de Meent and colleagues note that this can be seen as approximate compilation of a model into a program that samples from its posterior, and that amortized inference is also called "inference compilation" [1]. This is the same encoder-decoder structure used by the [variational autoencoder](https://aiwiki.ai/wiki/variational_autoencoder), which the Pyro paper uses as a worked example [5].

## Major systems

| System | Host or form | Introduced | Main inference | Latest release (as of November 2026) |
| --- | --- | --- | --- | --- |
| BUGS | Standalone declarative language | about 1989 [8] | Gibbs sampling [8] | n/a |
| Stan | Standalone language compiled to C++ | 2012 [10] | NUTS, ADVI, L-BFGS [3][19] | 2.40.0 (2026-09-16) [22] |
| PyMC | Python library on PyTensor | early 2000s [2] | NUTS, Metropolis, SMC, VI [2] | 6.4.0 (2026-11-01) [20] |
| Church | Scheme dialect | 2008 [4] | Monte Carlo [4] | n/a |
| Anglican | Embedded in Clojure | 2014 [11] | Particle MCMC [11] | n/a |
| Edward | Python library on TensorFlow | 2016 [13] | Variational inference, HMC [13] | 1.3.5 (2018-01-22) [23] |
| Pyro | Python library on PyTorch | 2017 [14] | SVI, NUTS [5] | 1.9.2 (2026-10-03) [24] |
| Turing.jl | Embedded in Julia | 2018 (AISTATS paper) [6] | HMC, particle Gibbs, composable [6] | 0.49.0 (2026-09-13) [25] |
| Gen | Julia library | 2019 (PLDI paper) [16] | Programmable inference [16] | Gen.jl 0.4.9 (2026-10-31) [26] |
| NumPyro | Python library on JAX | 2019 [15] | NUTS, SVI [15] | 0.22.0 (2026-09-18) [27] |

### Stan

Stan's documentation for version 2.40 covers MCMC sampling, optimization, Pathfinder, variational inference and Laplace approximation [28]. It has interfaces for Python, Julia, R and the Unix shell [29], and the 2017 paper listed further wrappers for MATLAB, Stata and Mathematica [3]. Stan is a NumFOCUS sponsored project, and its developers include both full-time staff and volunteers [9].

### PyMC

The 2023 PyMC paper, by Oriol Abril-Pla and 13 co-authors, describes a Python library whose syntax stays close to the notation statisticians use to describe models. PyMC compiles models through PyTensor to C, JAX or Numba backends, can run on CPUs, GPUs and TPUs, and supports hierarchical regression, time series, ordinary differential equations and [Gaussian process](https://aiwiki.ai/wiki/gaussian_process) models [2]. Besides its own samplers, it can call NUTS implementations from NumPyro and BlackJAX, which is possible because the modeling language is decoupled from the inference methods [2]. PyMC 6.0, released in May 2026, depends on PyTensor 3.0, makes Numba the default backend and uses the nutpie NUTS sampler by default when it is installed [20]. The project also incubated ArviZ, a library for diagnostics and analysis of Bayesian models [2].

### Pyro and NumPyro

Pyro was described by its authors as a PPL for developing advanced probabilistic models in AI research, using stochastic variational inference and distributions built on PyTorch to scale to large data sets and high-dimensional models [5]. Its 2019 paper listed an MIT license; the project's website now describes it as an Apache 2.0 licensed open-source project [5][30]. Pyro's developers also contributed upstream to PyTorch Distributions, a core PyTorch library inspired by TensorFlow Distributions [5]. NumPyro keeps Pyro's modeling interface but runs on JAX, and composing its effect handlers with JAX's just-in-time compilation, automatic differentiation and vectorization gives an iterative NUTS implementation that its authors report is substantially faster than existing alternatives on both small and large data sets [15].

### Gen

Gen was created by Marco Cusumano-Towner at the MIT Probabilistic Computing Project, led by Vikash Mansinghka [17]. The PLDI 2019 paper reports that Gen outperformed other PPLs, sometimes by several orders of magnitude, on problems including object tracking, estimating 3D body pose from a depth image, and inferring the structure of a time series [16]. Its main implementation is the Julia package Gen.jl [17].

### Other systems

Many other PPLs and probabilistic modeling libraries exist. The van de Meent textbook lists, among others, Hakaru, Augur, R2, Figaro, IBAL and PSI from programming-languages research; BLOG, BayesDB, Venture, Probabilistic-C and WebPPL from machine learning; and Biips, LibBi and Birch from statistics [1]. Microsoft's Infer.NET is a framework for Bayesian inference in graphical models that can also be used for probabilistic programming [31]. Google's TensorFlow Probability provides distributions, bijectors, joint distributions, probabilistic layers, MCMC and variational inference on top of TensorFlow [32].

## Applications

PPLs are used for ordinary statistical models, such as hierarchical regressions, as well as for models that are hard to write down in mathematical notation [2][1]. The van de Meent textbook gives several examples [1]:

- **Vision as inverse graphics.** A program that renders text images can be conditioned on a CAPTCHA image to infer a posterior over the hidden string, without hand-labeling a large training set. The authors stress that the point of this work was to show such models can be written and solved with general-purpose inference, not to break CAPTCHAs.
- **Constrained procedural graphics.** A stochastic program that grows trees can be conditioned so that branches avoid objects in a scene.
- **Program induction.** A generative model over program source code can be conditioned on input-output examples.
- **Multi-agent reasoning.** Church and WebPPL have been used to model agents that reason recursively about each other.
- **Scientific simulators.** An application under consideration when the text was written was to instrument the simulators of the Standard Model and of the detectors at the Large Hadron Collider, so that observing detector outputs would drive inference over the simulation.

The Stan paper's running example is a hierarchical model of tumor incidence in rats across trials, and it uses the closely related BUGS example of surgical mortality across hospitals to show how posterior rankings of hospitals are computed [3].

## Probabilistic programming and large language models

Probabilistic programming ideas have been applied to controlling [large language models](https://aiwiki.ai/wiki/large_language_model). In 2023, Alexander Lew, Tan Zhi-Xuan, Gabriel Grand and Vikash Mansinghka of MIT proposed sequential Monte Carlo (SMC) steering, which treats a constrained generation task as posterior inference and replaces standard decoding with SMC. They reported that, at a cost similar to [beam search](https://aiwiki.ai/wiki/beam_search), it could handle infilling, generation under syntactic constraints and "prompt intersection", and they released LLaMPPL, a library for writing such tasks as probabilistic programs that steer [LLaMA](https://aiwiki.ai/wiki/llama)-family Transformers [21]. A 2025 paper by João Loula and colleagues built on that framework for controlled generation in Python code generation, text-to-SQL, goal inference and molecule synthesis, reporting that small open-source models using the method outperformed models more than eight times larger [33].

## Limitations

Automatic inference trades generality for efficiency. The Pyro authors write that in practice advanced models often still need high-performance inference engines tailored to the application, and that their four design goals often pull against each other [5]. Exact inference in general programs is hard: the van de Meent textbook notes that methods exist to compile a probabilistic program into one that samples directly from its posterior only in special cases, and that in general this is difficult [1]. Stan's gradient-based MCMC targets models with continuous variables [3], and Stan's developers listed the lack of inference for discrete parameters as its main limitation from the user's side [10]. BUGS's developers also published a critical appraisal of how the flexibility of their system produced negative side effects [8].

## See also

- [Bayesian inference](https://aiwiki.ai/wiki/bayesian_inference)
- [Markov chain Monte Carlo](https://aiwiki.ai/wiki/markov_chain_monte_carlo)
- [Variational inference](https://aiwiki.ai/wiki/variational_inference)
- [Probabilistic graphical model](https://aiwiki.ai/wiki/probabilistic_graphical_model)
- [Generative model](https://aiwiki.ai/wiki/generative_model)
- [Prior belief](https://aiwiki.ai/wiki/prior_belief)

## References

1. Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, Frank Wood. "An Introduction to Probabilistic Programming". arXiv:1809.10756, 2018 (revised 2021). https://arxiv.org/abs/1809.10756
2. Oriol Abril-Pla, Virgile Andreani, Colin Carroll, Larry Dong, Christopher J. Fonnesbeck, Maxim Kochurov, Ravin Kumar, Junpeng Lao, Christian C. Luhmann, Osvaldo A. Martin, Michael Osthege, Ricardo Vieira, Thomas Wiecki, Robert Zinkov. "PyMC: a modern, and comprehensive probabilistic programming framework in Python". PeerJ Computer Science 9:e1516, 2023. https://doi.org/10.7717/peerj-cs.1516
3. Bob Carpenter, Andrew Gelman, Matthew D. Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, Allen Riddell. "Stan: A Probabilistic Programming Language". Journal of Statistical Software 76(1), 2017. https://doi.org/10.18637/jss.v076.i01
4. Noah D. Goodman, Vikash K. Mansinghka, Daniel M. Roy, Keith Bonawitz, Joshua B. Tenenbaum. "Church: a language for generative models". Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence (UAI), 2008, pp. 220-229. https://arxiv.org/abs/1206.3255
5. Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, Noah D. Goodman. "Pyro: Deep Universal Probabilistic Programming". Journal of Machine Learning Research 20(28):1-6, 2019. https://jmlr.org/papers/v20/18-403.html
6. Hong Ge, Kai Xu, Zoubin Ghahramani. "Turing: a language for flexible probabilistic inference". Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 84, 2018. https://proceedings.mlr.press/v84/ge18b.html
7. Stan Development Team. "Removed Features". Stan Reference Manual. https://mc-stan.org/docs/reference-manual/removals.html
8. David Lunn, David Spiegelhalter, Andrew Thomas, Nicky Best. "The BUGS project: Evolution, critique and future directions". Statistics in Medicine 28(25):3049-3067, 2009. https://doi.org/10.1002/sim.3680
9. Stan Development Team. "About Stan". mc-stan.org. https://mc-stan.org/about/
10. Andrew Gelman, Daniel Lee, Jiqiang Guo. "Stan: A Probabilistic Programming Language for Bayesian Inference and Optimization". Journal of Educational and Behavioral Statistics 40(5):530-543, 2015. https://doi.org/10.3102/1076998615606113
11. Frank Wood, Jan Willem van de Meent, Vikash Mansinghka. "A New Approach to Probabilistic Programming Inference". Proceedings of AISTATS 2014 (updated arXiv version, 2015). https://arxiv.org/abs/1507.00996
12. DARPA. "Probabilistic Programming for Advancing Machine Learning (PPAML)". darpa.mil. https://www.darpa.mil/research/programs/probabilistic-programming-for-advancing-machine-learning
13. Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, David M. Blei. "Edward: A library for probabilistic modeling, inference, and criticism". arXiv:1610.09787, 2016. https://arxiv.org/abs/1610.09787
14. Noah Goodman. "Uber AI Labs Open Sources Pyro, a Deep Probabilistic Programming Language". Uber Engineering Blog, 2017-11-02. https://www.uber.com/blog/pyro/
15. Du Phan, Neeraj Pradhan, Martin Jankowiak. "Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro". arXiv:1912.11554, 2019. https://arxiv.org/abs/1912.11554
16. Marco F. Cusumano-Towner, Feras A. Saad, Alexander K. Lew, Vikash K. Mansinghka. "Gen: a general-purpose probabilistic programming system with programmable inference". Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2019, pp. 221-236. https://doi.org/10.1145/3314221.3314642
17. MIT Probabilistic Computing Project. "Gen". gen.dev. https://www.gen.dev/
18. Matthew D. Hoffman, Andrew Gelman. "The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo". Journal of Machine Learning Research 15:1593-1623, 2014. https://jmlr.org/papers/v15/hoffman14a.html
19. Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, David M. Blei. "Automatic Differentiation Variational Inference". Journal of Machine Learning Research 18(14):1-45, 2017. https://jmlr.org/papers/v18/16-107.html
20. PyMC developers. "Releases: pymc-devs/pymc" (v4.0.0, v5.0.0, v6.0.0, v6.4.0). GitHub. https://github.com/pymc-devs/pymc/releases
21. Alexander K. Lew, Tan Zhi-Xuan, Gabriel Grand, Vikash K. Mansinghka. "Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs". arXiv:2306.03081, 2023. https://arxiv.org/abs/2306.03081
22. Stan Development Team. "Releases: stan-dev/stan" (v2.40.0). GitHub. https://github.com/stan-dev/stan/releases
23. Blei Lab. "Releases: blei-lab/edward". GitHub. https://github.com/blei-lab/edward/releases
24. Pyro developers. "Releases: pyro-ppl/pyro". GitHub. https://github.com/pyro-ppl/pyro/releases
25. TuringLang. "Releases: TuringLang/Turing.jl". GitHub. https://github.com/TuringLang/Turing.jl/releases
26. MIT Probabilistic Computing Project. "Releases: probcomp/Gen.jl". GitHub. https://github.com/probcomp/Gen.jl/releases
27. Pyro developers. "Releases: pyro-ppl/numpyro". GitHub. https://github.com/pyro-ppl/numpyro/releases
28. Stan Development Team. "Stan Reference Manual" (version 2.40). https://mc-stan.org/docs/reference-manual/
29. Stan Development Team. "Stan" (home page). mc-stan.org. https://mc-stan.org/
30. Pyro developers. "Pyro". pyro.ai. https://pyro.ai/
31. Microsoft. "Infer.NET" (README). GitHub. https://github.com/dotnet/infer
32. TensorFlow. "TensorFlow Probability" overview. tensorflow.org. https://www.tensorflow.org/probability/overview
33. João Loula, Benjamin LeBrun, Li Du, Ben Lipkin, Clemente Pasti, Gabriel Grand, Tianyu Liu, Yahya Emara, Marjorie Freedman, Jason Eisner, Ryan Cotterell, Vikash Mansinghka, Alexander K. Lew, Tim Vieira, Timothy J. O'Donnell. "Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo". arXiv:2504.13139, 2025. https://arxiv.org/abs/2504.13139

