# Control theory

> Source: https://aiwiki.ai/wiki/control_theory
> Updated: 2026-07-11
> Categories: Mathematics, Reinforcement Learning, Robotics
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution to "AI Wiki (aiwiki.ai)".

**Control theory** is the mathematical and engineering discipline concerned with designing and analysing systems that achieve desired behaviour through measurement and feedback. It studies how to manipulate the inputs (controls) to a dynamical system in order to make its outputs follow reference trajectories or stabilise around equilibria, in the presence of uncertainty and disturbances. The discipline is foundational for [robotics](/wiki/robotics), aerospace, the process industries, biology, economics, and increasingly for machine learning. Modern fields such as [reinforcement learning](/wiki/reinforcement_learning), [autonomous driving](/wiki/autonomous_driving), and humanoid manipulation all build on a control-theoretic substrate. The most widely deployed control algorithm is the humble PID (proportional-integral-derivative) controller: a Honeywell survey of more than 11,000 industrial controllers reported by Desborough and Miller found that 97 percent of regulatory loops use PID feedback [25].

The canonical modern reference is Karl Johan Aastrom and Richard M. Murray's *Feedback Systems: An Introduction for Scientists and Engineers* (Princeton University Press, 2nd ed. 2021), which won the 2011 IFAC Harold Chestnut Control Engineering Textbook Prize and is freely available online [1]. Other standard texts include Katsuhiko Ogata's *Modern Control Engineering* (5th ed., Pearson 2010) [2] and Hassan K. Khalil's *Nonlinear Systems* (3rd ed., Prentice Hall 2002) [3].

## Why does control theory matter for AI?

For a long time, control engineering and AI evolved on parallel tracks: control theorists assumed access to a model of the plant, while machine-learning researchers assumed access to data. The two fields have converged. Benjamin Recht's 2019 survey *A Tour of Reinforcement Learning: The View from Continuous Control* (Annual Review of Control, Robotics, and Autonomous Systems; arXiv:1806.09460) makes the case that reinforcement learning is best understood as data-driven optimal control, and that many open RL questions are restatements of long-studied control problems [4].

Several concrete reasons make control theory essential for modern AI:

* Reinforcement learning is, in many formulations, data-driven control. The Bellman equation that underlies value-function methods comes directly from Richard Bellman's 1957 dynamic programming, originally developed for optimal control [6].
* Robotics tightly intertwines control with perception and learning. Every quadruped, manipulator, and humanoid contains low-level control loops underneath any higher-level learned policy.
* Safety-critical AI systems often need control-theoretic guarantees such as Lyapunov stability, robust performance, or input-to-state stability that cannot be established by sample-based learning alone.
* Optimal control techniques such as the linear-quadratic regulator (LQR), the Hamilton-Jacobi-Bellman equation, and Pontryagin's maximum principle directly motivated the design of policy-gradient and actor-critic algorithms.
* Autonomous driving, drones, and humanoids embed control inside their stacks: even when a [reinforcement learning](/wiki/reinforcement_learning) policy chooses a high-level action, the low-level joint torques or wheel torques are still computed by a controller.

Richard Sutton and [Andrew Barto](/wiki/andrew_barto)'s *Reinforcement Learning: An Introduction* (2nd ed., MIT Press 2018) opens with an explicit acknowledgement that the dynamic-programming and optimal-control communities are one of the three intellectual sources of RL, alongside trial-and-error learning and temporal-difference learning [5].

## A brief history

Control theory has a long history that long predates electronic computers, let alone AI. Mechanical feedback regulators were used in antiquity (water clocks, float regulators), but the modern engineering discipline took shape in the late nineteenth century and matured in the twentieth.

| Year | Event |
|------|-------|
| ca. 250 BCE | Ktesibios of Alexandria designs a float-regulated water clock, an early example of mechanical feedback. |
| 1788 | James Watt and Matthew Boulton apply the centrifugal (flyball) governor to steam engines, regulating speed via feedback. |
| 1868 | James Clerk Maxwell publishes "On Governors" in the Proceedings of the Royal Society, the first formal stability analysis of a feedback regulator [7]. |
| 1892 | Aleksandr Lyapunov's doctoral dissertation establishes the general theory of stability of motion that bears his name. |
| 1922 | Nicholas Minorsky publishes the first analysis of three-term (PID) control for ship steering. |
| 1932 | Harry Nyquist at Bell Labs publishes "Regeneration Theory", giving the Nyquist stability criterion for feedback amplifiers [8]. |
| 1942 | John G. Ziegler and Nathaniel B. Nichols publish PID tuning rules at Taylor Instrument Companies. |
| 1945 | Hendrik W. Bode publishes *Network Analysis and Feedback Amplifier Design*, introducing Bode plots, gain margin, and phase margin [9]. |
| 1948 | Norbert Wiener publishes *Cybernetics: Or Control and Communication in the Animal and the Machine* (MIT Press), tying control to information theory and biology [10]. |
| 1957 | Richard Bellman publishes *Dynamic Programming* (Princeton), introducing the principle of optimality [6]. |
| 1960 | Rudolf E. [Kalman](/wiki/kalman) publishes "A New Approach to Linear Filtering and Prediction Problems" in the ASME Journal of Basic Engineering, introducing the Kalman filter [11]. The same year he formalises state-space optimal control (LQR) in *Boletin de la Sociedad Matematica Mexicana* [12]. |
| 1962 | Lev Pontryagin, V. Boltyanskii, R. Gamkrelidze, and E. Mishchenko publish *The Mathematical Theory of Optimal Processes* (Interscience), formalising the maximum principle [13]. |
| 1969 | Apollo 11 lunar module uses an onboard Kalman filter for descent navigation. |
| 1980s | Robust control develops: $$H_\infty$$, structured singular value (mu-synthesis), Quantitative Feedback Theory. Adaptive control matures (Aastrom and Wittenmark) [16]. |
| 1989 | John Doyle, Keith Glover, Pramod Khargonekar, and Bruce Francis publish "State-space solutions to standard H-2 and H-infinity control problems" in IEEE Transactions on Automatic Control [14]. |
| 1990s | Nonlinear control consolidates: feedback linearisation, sliding-mode (Utkin), backstepping (Krstic, Kanellakopoulos, Kokotovic). |
| 2000 | David Q. Mayne, James B. Rawlings, Christopher V. Rao, and Pierre O. M. Scokaert publish the survey "Constrained model predictive control: Stability and optimality" in *Automatica* [15]. |
| 2005 | Stanford's Stanley wins the DARPA Grand Challenge, demonstrating autonomous driving with classical estimation and control fused with machine learning [18]. |
| 2010s | Intersection with machine learning intensifies: data-driven control, learning-based [model predictive control](/wiki/model_predictive_control), differentiable MPC, model-free and model-based RL applied to robotics. |
| 2015 | SpaceX lands the Falcon 9 first stage using convex-optimisation guidance, an industrial demonstration of real-time optimal control. |
| 2020s | Whole-body MPC running at hundreds of hertz becomes standard on quadrupeds and humanoids (Boston Dynamics Atlas, ANYmal, Unitree). Foundation models start to appear inside control stacks. |

Norbert Wiener's revival of Maxwell's 1868 paper in *Cybernetics* (1948) is what cemented Maxwell's status as the founder of modern stability analysis [10]. Before Wiener pointed to it, the paper had been largely forgotten because contemporaries found it hard to read.

## What are the core concepts in control theory?

Most of the vocabulary used in control papers and robotics codebases comes from a small set of core ideas. The table below collects the most important ones; full treatments are in Aastrom and Murray (2021) [1] and Ogata (2010) [2].

| Concept | Meaning |
|---------|---------|
| Plant | The physical system being controlled (motor, aircraft, chemical reactor, robot arm). |
| Controller | The algorithm that computes the input to the plant from measurements. |
| Sensor | Device that produces a measurement of the plant state or output. |
| Actuator | Device that applies the control input (motor, valve, thruster). |
| Open-loop control | Control input is computed without using any measurement of the output. Sensitive to disturbances and model error. |
| Closed-loop (feedback) control | Control input is a function of measured output, allowing rejection of disturbances and tolerance to model error. |
| Feedforward | Control input is computed from a known reference or measured disturbance, often combined with feedback. |
| Reference tracking | Goal of making the output follow a desired trajectory. |
| Disturbance rejection | Goal of keeping the output near a setpoint despite external disturbances. |
| LTI system | [Linear](/wiki/linear), time-invariant system; the most studied class, where superposition holds and frequency-domain methods apply. |
| Transfer function | Ratio of Laplace-transformed output to input, describing an LTI system in the frequency domain. |
| State-space representation | Description of a system as $$\dot{x} = f(x, u)$$, $$y = h(x, u)$$, generalising to nonlinear and multi-input multi-output systems. |
| Laplace transform | Continuous-time integral transform used to convert ODEs into algebraic equations in the variable s. |
| z-transform | Discrete-time analogue of the Laplace transform, used for digital control. |
| Frequency response | The steady-state response of an LTI system to sinusoidal inputs, visualised via Bode and Nyquist plots. |
| BIBO stability | Bounded-input bounded-output stability of LTI systems; equivalent to all transfer-function poles being in the open left half-plane (continuous time). |
| Lyapunov stability | A solution is Lyapunov-stable if trajectories starting near it stay near it; asymptotic stability adds convergence. Proven via Lyapunov functions. |
| Controllability | A state-space system $$(A, B)$$ is controllable if its state can be driven from any initial value to any final value in finite time. Kalman's rank condition checks $$\text{rank}[B \; AB \; \cdots \; A^{n-1}B] = n$$ for LTI systems. |
| Observability | A system $$(A, C)$$ is observable if the initial state can be reconstructed from input and output histories. Dual to controllability. |
| Stabilisability and detectability | Weaker conditions than controllability and observability that suffice for stabilising controllers and stable observers. |

## What are the major control paradigms?

The field has accumulated many design paradigms, each suited to particular structure and assumptions. The table below summarises the main families.

| Paradigm | Key idea | Typical use |
|----------|----------|-------------|
| PID (Proportional-Integral-Derivative) | Output is a weighted sum of error, its integral, and its derivative. Three tunable gains. | The workhorse of industrial control. Surveys consistently report that around 90 to 97 percent of industrial control loops use PID; a Honeywell survey of more than 11,000 controllers reported by Desborough and Miller (2002) found 97 percent used PID feedback [25]. |
| State-feedback / pole placement | Design $$u = -Kx$$ so that the closed-loop matrix $$A - BK$$ has desired eigenvalues. | Multivariable LTI design when full state is measured or estimated. |
| Linear-quadratic regulator (LQR) | Optimal state feedback for linear dynamics and quadratic cost; gain K obtained by solving an algebraic Riccati equation. Introduced by Kalman (1960b) [12]. | Aerospace, helicopter and inverted-pendulum control, baseline for RL benchmarks. |
| Linear-quadratic-Gaussian (LQG) | LQR combined with a Kalman filter; satisfies a separation principle. | Standard for stochastic LTI systems with Gaussian noise. |
| Pontryagin's maximum principle | Necessary conditions for optimal control via a Hamiltonian; introduced by Pontryagin et al. (1962) [13]. | Trajectory optimisation, aerospace guidance. |
| Hamilton-Jacobi-Bellman (HJB) | Sufficient conditions and value-function PDE for continuous-time optimal control. | Theoretical foundation for dynamic programming and RL. |
| Robust control ($$H_\infty$$, mu-synthesis) | Minimise worst-case sensitivity to bounded disturbances and model uncertainty. State-space $$H_\infty$$ solution by Doyle, Glover, Khargonekar, Francis (1989) [14]. | Flight control, hard-disk read-write heads, applications with structured uncertainty. |
| Adaptive control | Controller parameters are adjusted online based on observed plant behaviour. | Plants whose parameters change slowly or are unknown, e.g. tail-rotor failures, fuel-burn shifts in aircraft. |
| [Nonlinear](/wiki/nonlinear) control | Techniques such as feedback linearisation, sliding-mode (Utkin 1977), backstepping (Krstic et al. 1995). | Robotic manipulators, electric drives, automotive subsystems. |
| [Model predictive control](/wiki/model_predictive_control) (MPC) | At each step, solve a finite-horizon optimal control problem and apply only the first input; repeat. Mayne et al. (2000) provide the canonical stability survey [15]. | Refineries since the 1980s, autonomous driving, quadrupeds, humanoids, rocket landings. |
| Sliding-mode control | Force the system state onto a designed sliding surface and keep it there using high-frequency switching. | Power converters, electric drives, automotive ABS. |
| Fuzzy control | Use linguistic if-then rules and fuzzy-set inference instead of explicit dynamics. | Consumer appliances, expert-elicited rule bases. |
| Reinforcement learning control | Learn a policy directly from interaction, model-free or model-based. | Game-playing, dexterous manipulation, locomotion, recently combined with classical control as warm starts and safety filters. |

PID dominates for a reason: it is robust, easy to tune, requires no plant model, and works well on near-linear loops. The more sophisticated paradigms become attractive when constraints, multivariable coupling, or strong nonlinearity break PID's assumptions.

## How do state observers and the Kalman filter work?

Most real systems do not give direct access to their internal state. Estimation theory bridges this gap by reconstructing state from noisy partial measurements. This is also the basis for [sensor fusion](/wiki/sensor_fusion).

The Kalman filter, introduced by Rudolf E. [Kalman](/wiki/kalman) in 1960, provides the optimal linear minimum-mean-square-error estimator for linear-Gaussian state-space systems [11]. It is recursive, updating its estimate as each new measurement arrives, and propagates a covariance matrix that quantifies uncertainty. The filter has been used continuously in aerospace since NASA's Stanley F. Schmidt adapted it for the Apollo program; every Apollo lunar mission, including the Apollo 11 descent in 1969, used a Kalman filter to fuse inertial measurement unit, optical, and ground-tracking data [27].

For nonlinear systems, the extended Kalman filter (EKF) linearises the dynamics and measurement equations around the current estimate. It is widely used despite well-known divergence problems on highly nonlinear systems. The unscented Kalman filter (UKF), introduced by Simon Julier and Jeffrey Uhlmann in 1997, propagates a deterministic set of sigma points through the nonlinear functions and reconstructs mean and covariance, often outperforming the EKF without computing Jacobians [17]. Particle filters use a Monte-Carlo set of weighted samples and apply to general non-Gaussian distributions; they are standard in [robotics](/wiki/robotics) for problems such as global localisation. Moving-horizon estimation is the dual of model predictive control on the estimation side, solving a constrained optimisation over a sliding window of past measurements.

In modern robot systems, the line between estimation and learning blurs. EKFs and UKFs sit inside SLAM systems such as MonoSLAM and FastSLAM; particle filters power Monte-Carlo localisation in mobile robots; learned visual feature detectors feed measurements into Kalman or factor-graph back ends.

## What are the applications in robotics and AI?

Control algorithms run, often invisibly, inside almost every autonomous system. The table below collects representative deployments.

| Domain | Typical control approach | Notes |
|--------|--------------------------|-------|
| Industrial process control | PID, MPC | Refineries, chemical plants, pulp and paper. MPC has been deployed on thousands of refinery columns since the 1980s (Aspen DMCplus, Honeywell Profit Controller). |
| HVAC and building automation | PID, supervisory MPC | Setpoint regulation for temperature, humidity, and CO2. |
| Power converters and grids | PID, sliding-mode, MPC | Inverters, DC-DC converters, frequency regulation. |
| Automotive subsystems | PID, sliding-mode, gain-scheduled control | Engine control, anti-lock braking (ABS), electronic stability programme (ESP), cruise control. |
| Aircraft autopilots | PID, gain-scheduled, robust $$H_\infty$$ | Boeing 777, Airbus A320 family, drones such as PX4 and ArduPilot stacks. |
| Spacecraft attitude and navigation | LQG, Kalman filter, sliding-mode | Apollo lunar module (Kalman filter), Mars rovers, ISS attitude control. |
| Rocket landing | Convex-optimisation guidance, MPC | SpaceX Falcon 9 first-stage powered descent uses real-time convex programming. |
| Industrial robot arms | PID for joint loops, computed-torque, MPC for trajectory | KUKA, ABB, FANUC; ROS Control framework. |
| Surgical robots | Cartesian impedance control, motion-scaling control | Intuitive Surgical's da Vinci system. |
| Drone flight control | Cascaded PID, geometric control on SO(3), MPC | PX4 and ArduPilot use cascaded PID by default; research stacks use geometric or MPC. |
| Autonomous driving | MPC for trajectory tracking, LQR for lateral control, EKF or UKF for state estimation | Stanford's Stanley (2005 DARPA Grand Challenge winner) used classical control plus learned terrain classifiers. Modern systems from Waymo, Cruise, and Tesla embed MPC for [motion planning](/wiki/motion_planning) and trajectory tracking. |
| Quadrupeds | Whole-body MPC, convex MPC, RL policies | Boston Dynamics Spot, ANYbotics ANYmal, MIT Cheetah, Unitree Go and B series. |
| Humanoids | Whole-body MPC, model-based optimisation | Boston Dynamics Atlas uses a model-predictive controller for whole-body motion; Tesla Optimus and Figure use similar internal control loops. |
| Manipulation | Operational-space control, impedance control, RL | Inner loops of dexterous-manipulation research stacks. |
| SLAM | EKF, UKF, particle filters, factor graphs | EKF-SLAM, MonoSLAM, FastSLAM use estimation-theory machinery for joint pose and map estimation. |

A particularly clean industrial example is the Falcon 9 booster landing. The vehicle solves a convex-optimisation problem in real time during powered descent, computing thrust profiles that satisfy fuel, attitude, and glide-slope constraints. The mathematics is descended directly from optimal-control theory developed in the 1960s.

## How does control theory relate to reinforcement learning?

The relationship between control and reinforcement learning is often summarised as: RL is control without a known model. Both fields care about choosing actions to optimise a long-horizon objective in a dynamical environment. Both rely on the Bellman equation, whose "principle of optimality" Bellman stated as: "An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision" [6]. The differences are in what is assumed and how solutions are computed.

Classical optimal control assumes a known dynamics model f and cost function g, then solves either Pontryagin's necessary conditions [13] (open-loop trajectory optimisation) or the Hamilton-Jacobi-Bellman equation (closed-loop value function). Reinforcement learning assumes that f and possibly g are unknown, but that the agent can interact with the environment and observe rewards. Value-function methods such as [Q-learning](/wiki/q_learning) and SARSA derive directly from the discrete-time Bellman equation; [policy-gradient](/wiki/policy_gradient) methods perform stochastic gradient ascent on the expected return; actor-critic methods combine the two.

Recht's 2019 survey is the cleanest bridge between the two literatures [4]. It analyses LQR with unknown dynamics, which Recht calls "perhaps the simplest and best-studied problem in optimal control", as the simplest non-trivial RL benchmark, derives sample-complexity bounds, and shows experimentally that model-based methods often dominate model-free methods on continuous-control problems where good models can be learned. Sutton and Barto's 2018 textbook makes the inverse point: trial-and-error RL existed before dynamic programming, and the synthesis of the two traditions defines modern RL [5].

Model-based RL (such as PILCO, Dyna, and Dreamer) is essentially MPC plus learned dynamics, with the planning step replaced by simulation in a learned [world model](/wiki/world_model). Differentiable MPC, introduced by Brandon Amos and colleagues in 2018, runs MPC inside the computation graph so that controller parameters can be trained by gradient descent [24]. Safe RL builds barrier-function or robust-MPC layers around learned policies to enforce constraints with provable guarantees.

## What is control as inference?

Control as inference reframes optimal control and reinforcement learning as probabilistic inference in a graphical model, so that choosing good actions becomes computing a posterior distribution rather than maximising a reward directly. The standard construction augments the usual state-action model with a binary "optimality" variable at each timestep, whose probability is set proportional to the exponentiated reward. Conditioning on optimality holding at every step and then inferring the distribution over actions recovers a control policy, and the messages passed during inference play the role of value functions.

The unifying modern reference is [Sergey Levine](/wiki/sergey_levine)'s 2018 tutorial *Reinforcement Learning and Control as Probabilistic Inference* (arXiv:1805.00909) [19]. Levine shows that a generalisation of the standard problem, "sometimes termed maximum entropy reinforcement learning, is equivalent to exact probabilistic inference in the case of deterministic dynamics, and variational inference in the case of stochastic dynamics" [19]. The payoff is methodological: "formalizing a problem as probabilistic inference," Levine writes, "in principle allows us to bring to bear a wide array of approximate inference tools, extend the model in flexible and powerful ways, and reason about compositionality and partial observability" [19].

The framework has deep roots in stochastic optimal control. Hilbert Kappen's path-integral control (2005) showed that a class of nonlinear stochastic control problems can be transformed into a linear equation solvable by sampling, with the control cost written as a Kullback-Leibler divergence between the controlled dynamics and the uncontrolled ("passive") dynamics [22]. Emanuel Todorov's linearly solvable Markov decision problems (2006 and 2009) made the same structure explicit for discrete problems: the Bellman equation becomes linear in the exponentiated value function, and optimal actions acquire a closed form [21]. Marc Toussaint's 2009 work recast robot trajectory optimisation as approximate inference, using message-passing methods that generalise the classical iterative LQG algorithm to non-Gaussian systems [23].

Maximum-entropy reinforcement learning is the practical face of this theory. Instead of maximising expected reward alone, a maximum-entropy agent maximises reward plus policy entropy, so that it aims, in the words of the [Soft Actor-Critic](/wiki/soft_actor_critic) paper, "to succeed at the task while acting as randomly as possible" [20]. Soft Actor-Critic (Haarnoja, Zhou, Abbeel, and Levine, 2018) turns this objective into an off-policy actor-critic algorithm and became one of the most widely used continuous-control methods in robotics [20]. The entropy bonus is exactly the KL-to-passive-dynamics cost of the control-as-inference view, which is why maximum-entropy RL inherits its favourable exploration and robustness properties.

Framing control as inference matters for AI because it lets a single approximate-inference toolbox (variational inference, expectation propagation, message passing, and the expectation-maximisation algorithm) serve both perception and control, and it clarifies why stochastic, entropy-regularised policies tend to explore better and transfer more robustly than their deterministic counterparts. It also connects control to the active-inference and free-energy accounts of decision making studied in computational neuroscience.

## Software

Control-system design is supported by mature toolchains in both academia and industry.

| Tool | Use |
|------|-----|
| MATLAB Control System Toolbox | Industry standard for transfer-function and state-space design, root locus, Bode and Nyquist plots, LQR, Kalman filter design. |
| Simulink | Block-diagram simulation and code generation; widely used in automotive and aerospace. |
| Python `python-control` package | Open-source MATLAB-style control library. |
| `scipy.signal` and `scipy.linalg` | LTI building blocks (state-space, transfer functions, Riccati solvers). |
| CasADi | Symbolic framework for nonlinear optimisation and optimal control. |
| ACADO and acados | Real-time MPC and nonlinear optimal control. |
| do-mpc | Python framework for MPC and moving-horizon estimation. |
| IPOPT | Interior-point nonlinear programming solver, often used as the inner solver for trajectory optimisation. |
| ROS Control framework | Standard hardware-abstraction and controller-manager layer in the Robot Operating System. |
| Drake (Toyota Research Institute) | Robotics-oriented modelling, simulation, optimisation, and control library written in C++ with Python bindings. |
| Mathematica | Symbolic control design and analysis. |

## Recent trends

The last decade has been defined by the convergence of classical control with machine learning. Several lines of work stand out:

* Learning-based MPC, where parts of the dynamics, cost, or terminal set are learned from data while preserving the receding-horizon structure that gives MPC its safety properties.
* Differentiable MPC and differentiable simulators, which let controllers be trained end-to-end with backpropagation [24].
* Safe RL with control-theoretic guarantees, using control-barrier functions, robust MPC safety filters, or Lyapunov-based shielding.
* Neural ODEs and continuous-time models meet control: neural networks parameterise the right-hand side of an ODE and are trained against trajectory data.
* Meta-learning for adaptive control, learning controllers that adjust to new dynamics from a few seconds of interaction.
* Foundation models for control: early research is exploring whether large pretrained models can be used as policies or world models for embodied agents.
* Verifiable and certifiable control with RL, applying formal-methods tools (HJ reachability, sum-of-squares, abstract interpretation) to learned policies.

What unifies these trends is the realisation that control and learning are not competing solutions to the same problem; they are complementary tools. Modern systems blend them at every layer of the stack.

## Notable demonstrations

A few demonstrations have shaped how researchers and the public understand the field:

* The Apollo program's lunar-module guidance and navigation, which combined an inertial platform, a Kalman filter, and explicit-guidance algorithms small enough to run on a 32-kilogram (70-pound) computer with about 2,000 words of erasable core memory (roughly 4 KB of read-write memory) plus 36,000 words of fixed rope memory [27].
* Stanford's Stanley winning the 2005 DARPA Grand Challenge by completing a roughly 132-mile desert course in 6 hours 53 minutes 58 seconds, at an average speed near 19 mph, fusing classical control and estimation with machine-learned terrain classifiers [18].
* [Boston Dynamics](/wiki/boston_dynamics)' Atlas performing parkour, backflips, and dynamic manipulation, all driven by a model-predictive controller with a learned add-on layer in newer versions.
* [SpaceX](/wiki/spacex)'s first successful Falcon 9 first-stage landing in December 2015, at Landing Zone 1, Cape Canaveral, the world's first recovery of an orbital-class rocket, repeated routinely since, demonstrating real-time convex-optimisation guidance at industrial scale.
* [DeepMind](/wiki/deepmind)'s data-centre cooling controllers, which used model-based optimisation against a learned model to cut Google data-centre cooling energy by around 40 percent (about a 15 percent reduction in overall power-usage-effectiveness overhead), as reported by Rich Evans and Jim Gao in 2016 [26].

## Connection to other engineering disciplines

Control theory does not stand alone. It draws on and feeds into several adjacent fields. Signal processing supplies filtering, sampling, and frequency-domain tools. Systems engineering supplies the methodology for integrating controllers into larger products. Optimisation supplies the algorithms (interior-point methods, sequential quadratic programming, convex programming) that power MPC and trajectory optimisation. Dynamical systems theory supplies the mathematics of stability, bifurcation, and chaos. Mechatronics fuses control with mechanical and electrical design for actuators and sensors. Information theory and statistics supply the foundations of estimation and Kalman filtering. The boundaries between these fields are porous, and modern researchers in robotics or autonomous systems generally need fluency in several of them.

For a modern engineer building [autonomous driving](/wiki/autonomous_driving) software, an [autonomous mobile robot](/wiki/autonomous_mobile_robot), or a humanoid platform, control theory is not optional background. It is the language in which the inner loops of the system are written, the mathematics in which safety arguments are framed, and the literature against which any new RL or learning-based method is benchmarked.

## References

1. Karl Johan Aastrom and Richard M. Murray, *Feedback Systems: An Introduction for Scientists and Engineers* (2nd ed., Princeton University Press, 2021). Winner of the 2011 IFAC Harold Chestnut Control Engineering Textbook Prize. Available at https://fbswiki.org
2. Katsuhiko Ogata, *Modern Control Engineering* (5th ed., Pearson, 2010).
3. Hassan K. Khalil, *Nonlinear Systems* (3rd ed., Prentice Hall, 2002).
4. Benjamin Recht, "A Tour of Reinforcement Learning: The View from Continuous Control", *Annual Review of Control, Robotics, and Autonomous Systems*, vol. 2, pp. 253-279, 2019. arXiv:1806.09460.
5. Richard S. Sutton and Andrew G. Barto, *Reinforcement Learning: An Introduction* (2nd ed., MIT Press, 2018).
6. Richard Bellman, *Dynamic Programming* (Princeton University Press, 1957).
7. James Clerk Maxwell, "On Governors", *Proceedings of the Royal Society of London*, vol. 16, pp. 270-283, 1868.
8. Harry Nyquist, "Regeneration Theory", *Bell System Technical Journal*, vol. 11, pp. 126-147, 1932.
9. Hendrik W. Bode, *Network Analysis and Feedback Amplifier Design* (Van Nostrand, 1945).
10. Norbert Wiener, *Cybernetics: Or Control and Communication in the Animal and the Machine* (MIT Press, 1948).
11. Rudolf E. Kalman, "A New Approach to Linear Filtering and Prediction Problems", *Transactions of the ASME, Journal of Basic Engineering*, vol. 82, ser. D, pp. 35-45, 1960.
12. Rudolf E. Kalman, "Contributions to the theory of optimal control", *Boletin de la Sociedad Matematica Mexicana*, vol. 5, pp. 102-119, 1960.
13. Lev S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko, *The Mathematical Theory of Optimal Processes* (Interscience, 1962).
14. John C. Doyle, Keith Glover, Pramod P. Khargonekar, and Bruce A. Francis, "State-space solutions to standard H-2 and H-infinity control problems", *IEEE Transactions on Automatic Control*, vol. 34, no. 8, pp. 831-847, 1989.
15. David Q. Mayne, James B. Rawlings, Christopher V. Rao, and Pierre O. M. Scokaert, "Constrained model predictive control: Stability and optimality", *Automatica*, vol. 36, no. 6, pp. 789-814, 2000.
16. Karl Johan Aastrom and Bjorn Wittenmark, *Adaptive Control* (2nd ed., Dover Publications, 2008).
17. Simon J. Julier and Jeffrey K. Uhlmann, "A New Extension of the Kalman Filter to Nonlinear Systems", *Proceedings of SPIE*, vol. 3068, 1997.
18. Sebastian Thrun et al., "Stanley: The Robot that Won the DARPA Grand Challenge", *Journal of Field Robotics*, vol. 23, no. 9, pp. 661-692, 2006.
19. Sergey Levine, "Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review", arXiv:1805.00909, 2018.
20. Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine, "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor", *Proceedings of the 35th International Conference on Machine Learning (ICML)*, pp. 1861-1870, 2018. arXiv:1801.01290.
21. Emanuel Todorov, "Efficient computation of optimal actions", *Proceedings of the National Academy of Sciences*, vol. 106, no. 28, pp. 11478-11483, 2009.
22. Hilbert J. Kappen, "Path integrals and symmetry breaking for optimal control theory", *Journal of Statistical Mechanics: Theory and Experiment*, P11011, 2005.
23. Marc Toussaint, "Robot trajectory optimization using approximate inference", *Proceedings of the 26th International Conference on Machine Learning (ICML)*, pp. 1049-1056, 2009.
24. Brandon Amos, Ivan Jimenez, Jacob Sacks, Byron Boots, and J. Zico Kolter, "Differentiable MPC for End-to-end Planning and Control", *Advances in Neural Information Processing Systems (NeurIPS) 31*, 2018.
25. Lane Desborough and Randy Miller, "Increasing Customer Value of Industrial Control Performance Monitoring: Honeywell's Experience", *AIChE Symposium Series (6th International Conference on Chemical Process Control)*, vol. 98, no. 326, pp. 169-189, 2002.
26. Richard Evans and Jim Gao, "DeepMind AI Reduces Google Data Centre Cooling Bill by 40%", DeepMind blog, 20 July 2016. https://deepmind.google/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-by-40/
27. Eldon C. Hall, *Journey to the Moon: The History of the Apollo Guidance Computer* (AIAA, 1996).