Probabilistic programming
Probabilistic programming is an approach to statistical modeling and machine learning in which a model is written as a computer program that makes random choices, and a general-purpose inference engine then works out which values of those random choices are consistent with observed data. A probabilistic programming language (PPL) adds two things to an ordinary language: a way to draw values at random from distributions, and a way to condition on observations. The program therefore describes a joint distribution over unknowns and data, and the system's job is to compute the conditional, or posterior, distribution of the unknowns given the data [1]. The field sits where statistics, machine learning and programming-language research meet, and its main promise is that the modeler writes the model while the system supplies the inference algorithm [1][2].
Well-known systems include Stan, PyMC, Pyro and its JAX-based sibling NumPyro, Turing.jl, Gen, and the Lisp-based research languages descended from Church [3][2][5][15][6][16][4]. Their inference engines draw on Markov chain Monte Carlo (especially Hamiltonian Monte Carlo and the No-U-Turn Sampler), variational inference, sequential Monte Carlo and importance sampling [1].
Core idea
In conventional programming, a program takes inputs and produces outputs. Statistical modeling runs in the other direction: it starts from observed data $y$, posits a generative model $p(x, y)$ for how hidden quantities $x$ produce that data, and asks for the posterior $p(x \mid y)$. Van de Meent, Paige, Yang and Wood describe probabilistic programming as doing Bayesian inference with the tools of computer science: a programming language denotes the model, and statistical inference algorithms compute the conditional distribution of the program's inputs that could have produced the observed output [1]. By Bayes' theorem,
where $p(x)$ is the prior, $p(y \mid x)$ is the likelihood and $p(y)$ is the normalizing constant (the evidence). A probabilistic program specifies the prior and likelihood implicitly through its sample and observe statements, and inference has to characterize the posterior over all the random choices made while the program runs [1].
What separates a PPL from an ordinary language with a random number generator is conditioning. Almost every language can simulate random values; a PPL also has syntax for declaring which values are observed and an evaluator that implements conditioning on them [1]. Van de Meent and colleagues call languages without such constructs "stochastic simulation languages" [1]. They also distinguish PPLs from graphical-model libraries in which ordinary code builds a model data structure: in a PPL the model is the one implied by running the program itself [1].
A minimal example
The first probabilistic program in the textbook by van de Meent and colleagues is a beta-Bernoulli coin model, written in a Lisp-like language similar to Anglican [1]:
(let [prior (beta a b)
x (sample prior)
likelihood (bernoulli x)
y 1]
(observe likelihood y)
x)
Run forward with the observe statement ignored, the program would just return a draw of the coin's bias x from the prior. The authors use it to explain the simplest kind of inference engine: run the program many times, and at each observe record a weight that is larger when the sampled x makes the observed outcome more likely. With a sensible choice of weight, the weighted samples form an asymptotically exact representation of the posterior. This is likelihood weighting, a form of importance sampling, and it requires running the program many times rather than once [1].
The same model in Stan declares data, parameters and a model block [3]:
data {
int<lower=0> N;
array[N] int<lower=0, upper=1> y;
}
parameters {
real<lower=0, upper=1> theta;
}
model {
theta ~ beta(1, 1);
y ~ bernoulli(theta);
}
The 2017 Stan paper printed this program with the older postfix array syntax (int<lower=0, upper=1> y[N];); that syntax was removed in Stan 2.33 in favor of the array[N] form shown here [3][7].
History
Statistical modeling languages
Statistics developed specialized languages for writing models and answering queries early on, among them BUGS and later Stan [1]. BUGS (Bayesian inference Using Gibbs Sampling) let users write a Bayesian model in a declarative language and fit it by Gibbs sampling. A 2009 retrospective by its developers, David Lunn, David Spiegelhalter, Andrew Thomas and Nicky Best, credited it with raising awareness of Bayesian modeling in academia and industry over a 20-year life span, and also offered a critical appraisal of its shortcomings [8]. JAGS (Plummer, 2003) followed the same declarative graphical-model approach [3].
Stan, developed by a team most of whose members were at Columbia University, released version 1.0 in 2012 [3][10]. It is named after Stanislaw Ulam, one of the developers of the Monte Carlo method in the 1940s [10]. The Stan team described two main departures from BUGS and JAGS: an imperative language with typed variables, local variables and conditional statements, and an MCMC engine based on Hamiltonian Monte Carlo rather than Gibbs sampling or Metropolis-Hastings [3].
Universal languages from machine learning and cognitive science
Church, presented at the 2008 Conference on Uncertainty in Artificial Intelligence by Noah Goodman, Vikash Mansinghka, Daniel Roy, Keith Bonawitz and Joshua Tenenbaum of MIT, was a "universal" language for stochastic generative processes [4]. It was based on a pure subset of Scheme, treated evaluation as sampling and its query primitive as conditional sampling, and introduced a stochastic memoizer that made many nonparametric models simple to write [4]. Its authors showed that queries on any Church program could be answered with Monte Carlo techniques [4]. The Pyro authors later described Church as an early universal PPL able to represent any computable probability distribution [5].
The Pyro authors describe Anglican and WebPPL as lightweight successors to Church, embedded in the general-purpose languages Clojure and JavaScript, and Venture as another universal language with its own syntax and virtual machine [5]. Anglican, introduced by Frank Wood, Jan-Willem van de Meent and Vikash Mansinghka at AISTATS 2014, used particle Markov chain Monte Carlo for inference in Turing-complete probabilistic programs, including programs with stochastic recursion [11]. Its later compiled version translates programs into Clojure through a continuation-passing-style transformation and runs on the Java Virtual Machine [11].
In March 2013 the US DARPA began the Probabilistic Programming for Advancing Machine Learning (PPAML) program, which aimed to greatly increase the number of people able to build machine learning applications. It was scheduled for 46 months in three phases through 2017, and DARPA lists it as complete [12].
Deep learning frameworks and probabilistic programming
Several later PPLs were built on deep learning frameworks, which gave them GPU support and automatic differentiation. Edward, by Dustin Tran, David Blei and colleagues at Columbia University, was a library for probabilistic modeling, inference and model criticism built on TensorFlow, with variational inference, Hamiltonian Monte Carlo and stochastic gradient Langevin dynamics among its algorithms [13]. In November 2017 Uber AI Labs released Pyro, a deep probabilistic programming language built on PyTorch, as open source [14]. Turing, a Julia PPL with composable inference, was presented at AISTATS 2018 [6]. NumPyro, a lightweight library that gives Pyro's modeling interface a NumPy-style backend on JAX, followed in 2019 [15]. Gen, from the MIT Probabilistic Computing Project, was presented at PLDI 2019 [16][17].
PyMC traces its history to the early 2000s. Version 2.0, released in January 2009, implemented Metropolis-Hastings sampling with distributions written in Fortran, and a 2016 rewrite (PyMC3) added gradient-based samplers including Hamiltonian Monte Carlo and NUTS on top of the Theano library [2]. After Theano's discontinuation was announced in 2017, PyMC moved to Theano's fork Aesara and then forked Aesara into PyTensor [2].
Timeline
| Year | Event |
|---|---|
| 2008 | Church presented at UAI [4] |
| 2009 | PyMC 2.0 released (January) [2] |
| 2009 | BUGS retrospective marks a 20-year life span [8] |
| 2012 | Stan 1.0 released [10] |
| 2013 | DARPA PPAML program begins (March) [12] |
| 2014 | Anglican presented at AISTATS; NUTS published in JMLR [11][18] |
| 2016 | Edward paper on arXiv; PyMC3 rewrite adds NUTS [13][2] |
| 2017 | Stan paper in Journal of Statistical Software; ADVI paper in JMLR; Pyro open-sourced (November) [3][19][14] |
| 2018 | Turing presented at AISTATS [6] |
| 2019 | Pyro paper in JMLR; Gen at PLDI; NumPyro paper on arXiv [5][16][15] |
| 2022 | PyMC 4.0 (June) and 5.0 (December) [20] |
| 2023 | LLaMPPL proposes SMC steering of LLMs with probabilistic programs [21] |
| 2026 | PyMC 6.0 (May); Stan 2.40 (September) [20][22] |
Language design
Static and dynamic models
Van de Meent and colleagues highlight one distinction as especially important, and structure their book around it: whether every program in a language denotes a model with a finite, fixed set of random variables [1]. In a "first-order" language without unbounded recursion, a program can be compiled into a finite graphical model (a Bayesian network or factor graph) and inference can use graph-based algorithms. In a "higher-order" language with recursion and dynamic control flow, the number of random variables can depend on the values of other random variables, and no such compilation is possible in general; inference must instead work by running the program [1]. BUGS-style languages and Stan sit on the static side, while Church, Anglican, WebPPL and Pyro allow dynamic control flow [1][5].
Standalone and embedded languages
Some PPLs are standalone languages with their own syntax and compiler. A Stan program is translated to C++ and compiled, and it imperatively defines a log probability function over the parameters given data and constants [3]. Its code is organized into named blocks: data, transformed data, parameters, transformed parameters, model and generated quantities [3]. Gelman, Lee and Guo noted in 2015 that, from the user's side, Stan's main limitation was that it did not allow inference for discrete parameters [10].
Others are embedded in a host language. Pyro programs are ordinary Python functions with two extra primitives, pyro.sample and pyro.param, that can be used with any Python control flow, including recursion, loops and conditionals [5]. The PyMC developers contrast three implementation strategies: PyMC builds an explicit computational graph in PyTensor, Pyro implements inference with effect handlers, and Stan compiles the program into a log-density function in C++ [2].
Design trade-offs
The Pyro paper frames PPL design around four goals that often conflict: expressivity (data-dependent control flow and latent variables whose existence depends on other variables), scalability (large data sets and high-dimensional models), flexibility (model-specific inference) and minimality (sharing syntax and semantics with existing languages) [5]. Its comparison table rates Stan as limited to static control flow but with automated inference, Church and Venture as fully expressive but not scalable, and Edward as scalable on CPU and GPU but limited to static control flow [5].
Inference methods
PPL inference engines are general algorithms that must work for any program a user can write in the language [1]. The main families are:
| Method | How it works | Example systems |
|---|---|---|
| Gibbs sampling | Updates variables one at a time from their conditional distributions | BUGS [8] |
| Metropolis-Hastings | Proposes changes to random choices and accepts or rejects them | Church, PyMC 2 [4][2] |
| Hamiltonian Monte Carlo, NUTS | Uses gradients of the log density to make long, informed moves | Stan, PyMC, Pyro, NumPyro, Turing [3][2][5][15][6] |
| Likelihood weighting, importance sampling | Runs the program forward and weights each run by the likelihood of the observations | General evaluators [1] |
| Sequential Monte Carlo, particle MCMC | Propagates and resamples a population of partial executions | Anglican, Turing, PyMC [11][6][2] |
| Variational inference | Fits an approximating distribution by optimization | Stan (ADVI), Pyro (SVI), PyMC [19][5][2] |
| Optimization | Finds a posterior mode or penalized maximum likelihood estimate | Stan (L-BFGS) [3] |
Hamiltonian Monte Carlo and NUTS
Hamiltonian Monte Carlo uses first-order gradient information to avoid the random-walk behavior of simpler samplers, but its performance depends on two user-set parameters: a step size and a number of steps. Matthew Hoffman and Andrew Gelman's No-U-Turn Sampler (NUTS), published in the Journal of Machine Learning Research in 2014, removes the need to set the number of steps by building a set of candidate points recursively and stopping when the trajectory starts to double back. Together with an adaptive step-size scheme, it can run with no hand-tuning, which the authors presented as suiting BUGS-style automatic inference engines [18]. Stan uses NUTS for MCMC, and NUTS is the default sampler for continuous variables in PyMC [10][2]. Because these methods need gradients, PPLs that use them rely on automatic differentiation: Stan through its C++ Stan Math Library for reverse-mode automatic differentiation, PyMC through PyTensor, Pyro through PyTorch and NumPyro through JAX [3][2][5][15].
Variational inference
Variational inference replaces sampling with optimization. With a family of distributions $q(x; \lambda)$ indexed by parameters $\lambda$, it maximizes the evidence lower bound (ELBO)
so that raising the bound pushes $q$ toward the posterior in KL divergence [1]. Automatic Differentiation Variational Inference (ADVI), by Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman and David Blei, derives a variational algorithm automatically from the model and data, needs no conjugacy assumptions, and was deployed as part of Stan [19]. Pyro's primary algorithm is gradient-based stochastic variational inference, which estimates the objective by Monte Carlo and scales to large data sets through mini-batches [5].
Programmable and composable inference
Automatic inference does not work well for every model. The Pyro authors note that advanced models often need inference engines tailored to the application [5], and the Gen paper argues that existing systems lacked the flexibility and efficiency needed for harder models in fields such as computer vision and robotics [16]. Two responses are composable and programmable inference. Turing combines MCMC operations on subsets of a model's variables, for example a Hamiltonian Monte Carlo engine for some variables and a particle Gibbs engine for others [6]. Gen introduces a generative function interface for encapsulating models, several interoperable modeling languages with different flexibility and efficiency trade-offs, combinators for common conditional-independence patterns, and an inference library for writing custom algorithms at a high level [16]. In Pyro, the guide (the approximate posterior or proposal used by algorithms such as SVI) is itself written as a Pyro program, and the Poutine library of effect handlers lets developers change how programs execute without touching the language internals [5].
Amortized inference
Standard algorithms solve one inference problem at a time. Amortized inference instead pays an up-front training cost to learn an inference network, usually a neural network, that maps a data set directly to the parameters of an approximate posterior, so that inference on new data is fast [1]. Van de Meent and colleagues note that this can be seen as approximate compilation of a model into a program that samples from its posterior, and that amortized inference is also called "inference compilation" [1]. This is the same encoder-decoder structure used by the variational autoencoder, which the Pyro paper uses as a worked example [5].
Major systems
| System | Host or form | Introduced | Main inference | Latest release (as of November 2026) |
|---|---|---|---|---|
| BUGS | Standalone declarative language | about 1989 [8] | Gibbs sampling [8] | n/a |
| Stan | Standalone language compiled to C++ | 2012 [10] | NUTS, ADVI, L-BFGS [3][19] | 2.40.0 (2026-09-16) [22] |
| PyMC | Python library on PyTensor | early 2000s [2] | NUTS, Metropolis, SMC, VI [2] | 6.4.0 (2026-11-01) [20] |
| Church | Scheme dialect | 2008 [4] | Monte Carlo [4] | n/a |
| Anglican | Embedded in Clojure | 2014 [11] | Particle MCMC [11] | n/a |
| Edward | Python library on TensorFlow | 2016 [13] | Variational inference, HMC [13] | 1.3.5 (2018-01-22) [23] |
| Pyro | Python library on PyTorch | 2017 [14] | SVI, NUTS [5] | 1.9.2 (2026-10-03) [24] |
| Turing.jl | Embedded in Julia | 2018 (AISTATS paper) [6] | HMC, particle Gibbs, composable [6] | 0.49.0 (2026-09-13) [25] |
| Gen | Julia library | 2019 (PLDI paper) [16] | Programmable inference [16] | Gen.jl 0.4.9 (2026-10-31) [26] |
| NumPyro | Python library on JAX | 2019 [15] | NUTS, SVI [15] | 0.22.0 (2026-09-18) [27] |
Stan
Stan's documentation for version 2.40 covers MCMC sampling, optimization, Pathfinder, variational inference and Laplace approximation [28]. It has interfaces for Python, Julia, R and the Unix shell [29], and the 2017 paper listed further wrappers for MATLAB, Stata and Mathematica [3]. Stan is a NumFOCUS sponsored project, and its developers include both full-time staff and volunteers [9].
PyMC
The 2023 PyMC paper, by Oriol Abril-Pla and 13 co-authors, describes a Python library whose syntax stays close to the notation statisticians use to describe models. PyMC compiles models through PyTensor to C, JAX or Numba backends, can run on CPUs, GPUs and TPUs, and supports hierarchical regression, time series, ordinary differential equations and Gaussian process models [2]. Besides its own samplers, it can call NUTS implementations from NumPyro and BlackJAX, which is possible because the modeling language is decoupled from the inference methods [2]. PyMC 6.0, released in May 2026, depends on PyTensor 3.0, makes Numba the default backend and uses the nutpie NUTS sampler by default when it is installed [20]. The project also incubated ArviZ, a library for diagnostics and analysis of Bayesian models [2].
Pyro and NumPyro
Pyro was described by its authors as a PPL for developing advanced probabilistic models in AI research, using stochastic variational inference and distributions built on PyTorch to scale to large data sets and high-dimensional models [5]. Its 2019 paper listed an MIT license; the project's website now describes it as an Apache 2.0 licensed open-source project [5][30]. Pyro's developers also contributed upstream to PyTorch Distributions, a core PyTorch library inspired by TensorFlow Distributions [5]. NumPyro keeps Pyro's modeling interface but runs on JAX, and composing its effect handlers with JAX's just-in-time compilation, automatic differentiation and vectorization gives an iterative NUTS implementation that its authors report is substantially faster than existing alternatives on both small and large data sets [15].
Gen
Gen was created by Marco Cusumano-Towner at the MIT Probabilistic Computing Project, led by Vikash Mansinghka [17]. The PLDI 2019 paper reports that Gen outperformed other PPLs, sometimes by several orders of magnitude, on problems including object tracking, estimating 3D body pose from a depth image, and inferring the structure of a time series [16]. Its main implementation is the Julia package Gen.jl [17].
Other systems
Many other PPLs and probabilistic modeling libraries exist. The van de Meent textbook lists, among others, Hakaru, Augur, R2, Figaro, IBAL and PSI from programming-languages research; BLOG, BayesDB, Venture, Probabilistic-C and WebPPL from machine learning; and Biips, LibBi and Birch from statistics [1]. Microsoft's Infer.NET is a framework for Bayesian inference in graphical models that can also be used for probabilistic programming [31]. Google's TensorFlow Probability provides distributions, bijectors, joint distributions, probabilistic layers, MCMC and variational inference on top of TensorFlow [32].
Applications
PPLs are used for ordinary statistical models, such as hierarchical regressions, as well as for models that are hard to write down in mathematical notation [2][1]. The van de Meent textbook gives several examples [1]:
- Vision as inverse graphics. A program that renders text images can be conditioned on a CAPTCHA image to infer a posterior over the hidden string, without hand-labeling a large training set. The authors stress that the point of this work was to show such models can be written and solved with general-purpose inference, not to break CAPTCHAs.
- Constrained procedural graphics. A stochastic program that grows trees can be conditioned so that branches avoid objects in a scene.
- Program induction. A generative model over program source code can be conditioned on input-output examples.
- Multi-agent reasoning. Church and WebPPL have been used to model agents that reason recursively about each other.
- Scientific simulators. An application under consideration when the text was written was to instrument the simulators of the Standard Model and of the detectors at the Large Hadron Collider, so that observing detector outputs would drive inference over the simulation.
The Stan paper's running example is a hierarchical model of tumor incidence in rats across trials, and it uses the closely related BUGS example of surgical mortality across hospitals to show how posterior rankings of hospitals are computed [3].
Probabilistic programming and large language models
Probabilistic programming ideas have been applied to controlling large language models. In 2023, Alexander Lew, Tan Zhi-Xuan, Gabriel Grand and Vikash Mansinghka of MIT proposed sequential Monte Carlo (SMC) steering, which treats a constrained generation task as posterior inference and replaces standard decoding with SMC. They reported that, at a cost similar to beam search, it could handle infilling, generation under syntactic constraints and "prompt intersection", and they released LLaMPPL, a library for writing such tasks as probabilistic programs that steer LLaMA-family Transformers [21]. A 2025 paper by João Loula and colleagues built on that framework for controlled generation in Python code generation, text-to-SQL, goal inference and molecule synthesis, reporting that small open-source models using the method outperformed models more than eight times larger [33].
Limitations
Automatic inference trades generality for efficiency. The Pyro authors write that in practice advanced models often still need high-performance inference engines tailored to the application, and that their four design goals often pull against each other [5]. Exact inference in general programs is hard: the van de Meent textbook notes that methods exist to compile a probabilistic program into one that samples directly from its posterior only in special cases, and that in general this is difficult [1]. Stan's gradient-based MCMC targets models with continuous variables [3], and Stan's developers listed the lack of inference for discrete parameters as its main limitation from the user's side [10]. BUGS's developers also published a critical appraisal of how the flexibility of their system produced negative side effects [8].
Continue reading
- Variational InferenceVariational inference (VI), also called variational Bayes (VB), is a method in machine learning and statistics that approximates an intractable posterior distribution by turning Bayesian inference into an…Mentioned 11 times above
- Bayesian inferenceBayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability of a hypothesis as new evidence or data becomes available.Mentioned 3 times above
- Markov Chain Monte CarloMarkov Chain Monte Carlo (MCMC) is a class of algorithms for drawing samples from a probability distribution by constructing a Markov chain whose stationary distribution equals the target distribution.Mentioned twice above
- Generative ModelA generative model is a class of statistical and machine learning model that learns the joint probability distribution P(X) of the observed data, or the joint distribution P(X, Y) of inputs and labelsMentioned twice above
- Probabilistic Graphical ModelA probabilistic graphical model (PGM) is a mathematical representation in which a graph expresses the factorization and conditional-independence structure of a probability distribution.Listed in See also
- Prior beliefA prior belief, also called the prior distribution or simply the prior, is the probability distribution that expresses what is known or assumed about an unknown parameter before any data is observed.Listed in See also
More to read
References
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, Frank Wood. "An Introduction to Probabilistic Programming". arXiv:1809.10756, 2018 (revised 2021). arxiv.org/...1809.10756
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19Oriol Abril-Pla, Virgile Andreani, Colin Carroll, Larry Dong, Christopher J. Fonnesbeck, Maxim Kochurov, Ravin Kumar, Junpeng Lao, Christian C. Luhmann, Osvaldo A. Martin, Michael Osthege, Ricardo Vieira, Thomas Wiecki, Robert Zinkov. "PyMC: a modern, and comprehensive probabilistic programming framework in Python". PeerJ Computer Science 9:e1516, 2023. doi.org/...peerj-cs.1516
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16Bob Carpenter, Andrew Gelman, Matthew D. Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, Allen Riddell. "Stan: A Probabilistic Programming Language". Journal of Statistical Software 76(1), 2017. doi.org/...jss.v076.i01
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Noah D. Goodman, Vikash K. Mansinghka, Daniel M. Roy, Keith Bonawitz, Joshua B. Tenenbaum. "Church: a language for generative models". Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence (UAI), 2008, pp. 220-229. arxiv.org/...1206.3255
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, Noah D. Goodman. "Pyro: Deep Universal Probabilistic Programming". Journal of Machine Learning Research 20(28):1-6, 2019. jmlr.org/...18-403
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Hong Ge, Kai Xu, Zoubin Ghahramani. "Turing: a language for flexible probabilistic inference". Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 84, 2018. proceedings.mlr.press/...ge18b
- ^Stan Development Team. "Removed Features". Stan Reference Manual. mc-stan.org/...removals
- ^1 ^2 ^3 ^4 ^5 ^6David Lunn, David Spiegelhalter, Andrew Thomas, Nicky Best. "The BUGS project: Evolution, critique and future directions". Statistics in Medicine 28(25):3049-3067, 2009. doi.org/...sim.3680
- ^Stan Development Team. "About Stan". mc-stan.org. mc-stan.org/about
- ^1 ^2 ^3 ^4 ^5 ^6 ^7Andrew Gelman, Daniel Lee, Jiqiang Guo. "Stan: A Probabilistic Programming Language for Bayesian Inference and Optimization". Journal of Educational and Behavioral Statistics 40(5):530-543, 2015. doi.org/...1076998615606113
- ^1 ^2 ^3 ^4 ^5 ^6Frank Wood, Jan Willem van de Meent, Vikash Mansinghka. "A New Approach to Probabilistic Programming Inference". Proceedings of AISTATS 2014 (updated arXiv version, 2015). arxiv.org/...1507.00996
- ^1 ^2DARPA. "Probabilistic Programming for Advancing Machine Learning (PPAML)". darpa.mil. darpa.mil/...amming-for-advancing-machine-learning
- ^1 ^2 ^3 ^4Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, David M. Blei. "Edward: A library for probabilistic modeling, inference, and criticism". arXiv:1610.09787, 2016. arxiv.org/...1610.09787
- ^1 ^2 ^3Noah Goodman. "Uber AI Labs Open Sources Pyro, a Deep Probabilistic Programming Language". Uber Engineering Blog, 2017-11-02. uber.com/...pyro
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Du Phan, Neeraj Pradhan, Martin Jankowiak. "Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro". arXiv:1912.11554, 2019. arxiv.org/...1912.11554
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Marco F. Cusumano-Towner, Feras A. Saad, Alexander K. Lew, Vikash K. Mansinghka. "Gen: a general-purpose probabilistic programming system with programmable inference". Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2019, pp. 221-236. doi.org/...3314221.3314642
- ^1 ^2 ^3MIT Probabilistic Computing Project. "Gen". gen.dev. gen.dev
- ^1 ^2Matthew D. Hoffman, Andrew Gelman. "The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo". Journal of Machine Learning Research 15:1593-1623, 2014. jmlr.org/...hoffman14a
- ^1 ^2 ^3 ^4Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, David M. Blei. "Automatic Differentiation Variational Inference". Journal of Machine Learning Research 18(14):1-45, 2017. jmlr.org/...16-107
- ^1 ^2 ^3 ^4PyMC developers. "Releases: pymc-devs/pymc" (v4.0.0, v5.0.0, v6.0.0, v6.4.0). GitHub. github.com/...releases
- ^1 ^2Alexander K. Lew, Tan Zhi-Xuan, Gabriel Grand, Vikash K. Mansinghka. "Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs". arXiv:2306.03081, 2023. arxiv.org/...2306.03081
- ^1 ^2Stan Development Team. "Releases: stan-dev/stan" (v2.40.0). GitHub. github.com/...releases
- ^Blei Lab. "Releases: blei-lab/edward". GitHub. github.com/...releases
- ^Pyro developers. "Releases: pyro-ppl/pyro". GitHub. github.com/...releases
- ^TuringLang. "Releases: TuringLang/Turing.jl". GitHub. github.com/...releases
- ^MIT Probabilistic Computing Project. "Releases: probcomp/Gen.jl". GitHub. github.com/...releases
- ^Pyro developers. "Releases: pyro-ppl/numpyro". GitHub. github.com/...releases
- ^Stan Development Team. "Stan Reference Manual" (version 2.40). mc-stan.org/...reference-manual
- ^Stan Development Team. "Stan" (home page). mc-stan.org. mc-stan.org
- ^Pyro developers. "Pyro". pyro.ai. pyro.ai
- ^Microsoft. "Infer.NET" (README). GitHub. github.com/...infer
- ^TensorFlow. "TensorFlow Probability" overview. tensorflow.org. tensorflow.org/...overview
- ^João Loula, Benjamin LeBrun, Li Du, Ben Lipkin, Clemente Pasti, Gabriel Grand, Tianyu Liu, Yahya Emara, Marjorie Freedman, Jason Eisner, Ryan Cotterell, Vikash Mansinghka, Alexander K. Lew, Tim Vieira, Timothy J. O'Donnell. "Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo". arXiv:2504.13139, 2025. arxiv.org/...2504.13139
Improve this article
Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.
1 revision · v1 · 4,376 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify
Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here
Reviewer note: Independent fact-check before publication on November 18, 2026: 118 claims checked against 33 cited and independent sources; 1 material and 7 minor corrections applied before publication.
Cite this page: AI Wiki. "Probabilistic programming." aiwiki.ai, updated 18 Nov 2026, fact-checked 18 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/probabilistic_programming

