Root Mean Squared Error (RMSE)

See also: Mean Squared Error (MSE), Mean Absolute Error (MAE)

What is Root Mean Squared Error (RMSE)?

Root Mean Squared Error (RMSE), also known as root mean square deviation (RMSD), is a regression evaluation metric equal to the square root of the average of the squared differences between predicted and observed values.^[1] Because it takes the square root of the mean squared error (MSE), RMSE is expressed in the same units as the target variable, which makes it directly interpretable as a typical prediction error.^[5] By squaring the errors before averaging, RMSE assigns disproportionate weight to large errors, so it is more sensitive to outliers than the mean absolute error (MAE).^[2][3]

RMSE is one of the most widely used metrics for evaluating the performance of regression models in machine learning, statistics, and the applied sciences.^[1] Its most famous large-scale use was the Netflix Prize (2006 to 2009), which set RMSE as the single competition criterion: the $1 million grand prize required cutting the RMSE of movie-rating predictions by 10% relative to Netflix's existing Cinematch algorithm, whose benchmark RMSE on the test set was 0.9525.^[14] The winning team, BellKor's Pragmatic Chaos, reached an RMSE of 0.8567 on 26 July 2009, a 10.06% improvement over Cinematch.^[14]

RMSE is closely related to MSE: it is simply the square root of MSE. While MSE is often preferred as a loss function during model training because it is differentiable and well-suited for gradient descent, RMSE is preferred for reporting and interpretation because its units match the target variable.^[5] Minimizing RMSE is mathematically equivalent to minimizing MSE, since the square root function is monotonically increasing and preserves the ranking of models.

RMSE can also be understood as the standard deviation of the prediction errors (residuals) when the mean error is zero. In this sense, RMSE describes how spread out the residuals are around the predicted values.

How is RMSE calculated? (Mathematical definition)

Given a dataset of n observations, where y_i is the actual observed value and ŷ_i (y-hat) is the predicted value for the i-th observation, the Root Mean Squared Error is defined as:

RMSE = √[(1/n) Σᵢ₌₁ⁿ (yᵢ − ŷᵢ)²]

In expanded form:

RMSE = √[(1/n) ((y₁ − ŷ₁)² + (y₂ − ŷ₂)² + ... + (yₙ − ŷₙ)²)]

The formula proceeds in three steps:

1. Compute the residual (error) for each observation: eᵢ = yᵢ − ŷᵢ
2. Square each residual, sum them, and divide by n to get the mean squared error.
3. Take the square root of MSE to bring the result back to the original units of measurement.

Scroll sideways for more →

Estimator form

In estimation theory, for an estimator θ̂ of a true parameter θ, the RMSE is defined as:

RMSE(θ̂) = √[E((θ̂ − θ)²)] = √MSE(θ̂)

For an unbiased estimator (where the bias is zero), the RMSE is equal to the standard deviation of the estimator, also called the standard error.^[1]

Worked example

Suppose a model makes predictions for five data points:

Scroll sideways for more →

Sum of squared errors = 0.25 + 0.25 + 0.00 + 1.00 + 0.25 = 1.75

MSE = 1.75 / 5 = 0.35

RMSE = √0.35 ≈ 0.592

This tells us the model's predictions are off by roughly 0.59 units on average in terms of the root-mean-square deviation, using the same units as the target variable.

Key properties

RMSE has several mathematical properties that make it a standard metric across scientific and engineering disciplines.

Scroll sideways for more →

How do you interpret an RMSE value?

Interpreting an RMSE value requires context. There is no universal threshold that separates a "good" from a "bad" RMSE, because the metric is scale-dependent. The same RMSE value can be excellent in one application and poor in another.

Guidelines for interpretation

Compare to the range or mean of observed values. An RMSE of 5 is very different depending on whether the target variable ranges from 0 to 10 (RMSE is 50% of the range) or from 0 to 10,000 (RMSE is 0.05% of the range). Normalizing RMSE by the range, mean, or standard deviation of the observed data (see the Normalized RMSE section below) provides a scale-free measure for comparison.^[8]

Compare to a baseline model. A useful benchmark is to compare RMSE against a naive model such as one that always predicts the mean of the training data. If the model's RMSE is substantially lower than the baseline's RMSE, it has learned meaningful patterns.

Approximate confidence interval. If the residuals are approximately normally distributed, roughly 68% of predictions fall within +/- 1 RMSE of the true value, and about 95% of predictions fall within +/- 2 RMSE of the true value. This property follows from the relationship between RMSE and the standard deviation of the residuals.^[9]

Consider the application. In safety-critical applications (medical dosing, structural engineering), even a small RMSE might be unacceptable. In exploratory forecasting (long-range weather prediction, economic modeling), a larger RMSE may be perfectly reasonable.

Bias-variance decomposition

The squared RMSE (i.e., MSE) can be decomposed into bias and variance components:

RMSE² = MSE = Bias² + Variance + σ²

Where:

• Bias² measures the systematic error: the squared difference between the expected prediction and the true value. High bias indicates underfitting.
• Variance measures the sensitivity of predictions to different training sets. High variance indicates overfitting.
• σ² is the irreducible noise in the data that no model can capture.

This decomposition reveals that reducing RMSE requires managing the bias-variance tradeoff.^[9] A model that is too simple will have high bias, while a model that is too complex will have high variance. Both scenarios inflate RMSE.^[1]

For model diagnostics, MSE can also be decomposed into systematic and unsystematic components.^[12] The systematic component (MSE_s) captures consistent over- or under-prediction and non-unity regression slopes. The unsystematic component (MSE_u) captures random scatter around the regression line. A well-calibrated model should have most of its MSE in the unsystematic component.^[11]

Connection to maximum likelihood estimation

Minimizing RMSE (or equivalently MSE) is mathematically equivalent to maximum likelihood estimation when the errors follow a Gaussian (normal) distribution.^[5] Under the assumption that yᵢ = f(xᵢ; θ) + εᵢ where εᵢ ~ N(0, σ²), the log-likelihood is:

log L(θ) = -(n/2) log(2πσ²) - (1/(2σ²)) Σ(yᵢ − f(xᵢ; θ))²

Maximizing this expression is equivalent to minimizing the sum of squared errors.^[6] This provides the theoretical justification for using RMSE as the primary evaluation metric when errors are normally distributed (Hodson, 2022).^[4]

Conversely, when errors follow a Laplacian distribution (heavier tails, more outlier-prone), minimizing MAE is the optimal approach under maximum likelihood.^[4] The choice between RMSE and MAE therefore depends fundamentally on the error distribution of the data.

RMSE vs. MSE: what is the difference?

RMSE and MSE are closely related but serve different purposes:

Scroll sideways for more →

An RMSE of 10 does not mean the model is off by 10 units on average. Because errors are squared before averaging, RMSE gives more weight to larger errors. The mean absolute error (MAE) provides the true average absolute error.

RMSE vs. MAE: what is the difference?

The relationship between RMSE and MAE has been the subject of extensive discussion in the scientific literature. Key differences are:

Scroll sideways for more →

The inequality MAE ≤ RMSE always holds. The two metrics are equal only when all errors are identical in magnitude. The upper bound RMSE ≤ √n · MAE is reached when all of the error is concentrated in a single observation.^[3]

Why is RMSE sensitive to outliers?

RMSE is sensitive to outliers because each residual is squared before the values are averaged, so a single large error grows quadratically while small errors stay small. For example, an error of 10 contributes 100 to the sum of squared errors, whereas an error of 1 contributes only 1, so one large miss can dominate the metric. MAE, which sums the absolute errors linearly, treats that same error of 10 as only ten times an error of 1, which is why MAE is far more robust to outliers and heavy-tailed error distributions.^[2][3] This is the central tradeoff: RMSE rewards models that avoid catastrophic individual errors, while MAE reflects the typical error magnitude without letting a few extreme cases take over.

The RMSE vs. MAE debate

This debate has a notable history in the geosciences. Willmott and Matsuura (2005) argued that RMSE is not a good indicator of average model performance because it varies with the variability of the error distribution, not just the average error magnitude. They recommended MAE as a more natural and unambiguous measure of average error, concluding that "the MAE is a more natural measure of average error, and (unlike the RMSE) is unambiguous."^[2]

Chai and Draxler (2014) responded with a counterargument, demonstrating that RMSE is not ambiguous in its meaning and is more appropriate when the error distribution is expected to be Gaussian. As they put it, "the RMSE is more appropriate to represent model performance than the MAE when the error distribution is expected to be Gaussian."^[3] They also showed that RMSE satisfies the triangle inequality, qualifying it as a proper distance metric.

Hodson (2022) provided a resolution to this debate by demonstrating that neither metric is inherently superior. The appropriate choice depends on the underlying error distribution: RMSE is optimal for Gaussian errors, and MAE is optimal for Laplacian errors. Hodson recommended that practitioners analyze the distributional properties of their errors before selecting a metric, rather than defaulting to convention.^[4]

When to use RMSE over MAE

Scroll sideways for more →

Normalized RMSE (NRMSE)

Because RMSE is scale-dependent, comparing models across datasets with different scales requires normalization.^[8] Normalized RMSE (NRMSE) divides RMSE by a characteristic value of the observed data. There are several common normalization methods:

Scroll sideways for more →

NRMSE is dimensionless and typically expressed as a proportion or percentage. Lower values indicate better model fit. A range-normalized NRMSE below 0.1 (10%) is generally considered good, though this depends heavily on the application domain.

There is no single normalization method that is universally superior. The choice depends on the data characteristics and the purpose of the comparison.

RMSE as a loss function

While RMSE is most commonly used as an evaluation metric, it can also serve as a loss function. However, MSE (the square of RMSE) is more commonly used for optimization because:

1. The gradient of MSE with respect to predictions is simpler: ∂MSE/∂ŷᵢ = -(2/n)(yᵢ − ŷᵢ)
2. The additional square root in RMSE introduces an extra scaling factor in the gradient without changing which parameter values minimize the loss.
3. Since RMSE and MSE rank models identically, there is no practical reason to use RMSE during optimization.

For this reason, MSE is the standard regression loss function during training in frameworks such as PyTorch, TensorFlow, and scikit-learn, while RMSE is reported after training to communicate model performance in interpretable units.^[5]

RMSE vs. other regression metrics

RMSE is one of several metrics used to evaluate regression models. The table below compares the most common alternatives:

Scroll sideways for more →

No single metric captures all aspects of model performance. It is common practice to report RMSE alongside R² and/or MAE to provide a more complete picture.^[9]

What is RMSE used for? (Applications)

RMSE is used across a broad range of scientific and engineering disciplines. Its versatility comes from the combination of intuitive units, mathematical tractability, and sensitivity to large errors.

Recommender systems and the Netflix Prize

The Netflix Prize competition (2 October 2006 to 26 July 2009) used RMSE as its sole evaluation criterion. Competitors aimed to reduce the RMSE of movie-rating predictions by at least 10% compared to Netflix's existing Cinematch algorithm, which scored an RMSE of 0.9525 on the test set.^[14] After three years of effort, the team BellKor's Pragmatic Chaos won the $1 million grand prize with a test-set RMSE of 0.8567, a 10.06% improvement over Cinematch.^[14] The competition is widely credited with popularizing matrix-factorization and ensemble methods, and RMSE remains a default scoring metric for regression problems on platforms such as Kaggle.

Weather forecasting and climate science

RMSE is a standard evaluation metric for numerical weather prediction (NWP) models.^[12] Environment Canada and other meteorological agencies use latitude-weighted RMSE to assess forecast accuracy for temperature, pressure, wind speed, and other variables at different lead times. In ensemble forecasting, comparing the RMSE of the ensemble mean against the ensemble spread provides a diagnostic for whether the ensemble is properly calibrated.

In climate modeling, RMSE is used to evaluate downscaling methods, general circulation model outputs, and data-driven forecasting approaches such as those benchmarked in ClimateLearn (Nguyen et al., 2023).^[13]

Bioinformatics and drug design

In structural bioinformatics, RMSD measures the average distance between atoms of superimposed protein structures. This application is central to protein structure prediction, molecular docking, and drug design, where the RMSD between a crystallographic conformation and a predicted pose indicates prediction quality.

Engineering and control systems

In control theory, RMSE (or RMSD) serves as a quality measure for state observer performance. In fluid dynamics, it quantifies flow uniformity for velocity, temperature, and concentration fields. In building energy simulation, RMSE is used to calibrate predicted energy consumption against measured data.

Remote sensing and GIS

RMSE is used to assess accuracy in spatial analysis, digital elevation models, and satellite imagery classification. In hydrogeology, it is the primary metric for calibrating groundwater flow models.

Image quality assessment

In imaging science, RMSE is a component of the Peak Signal-to-Noise Ratio (PSNR), which is widely used to measure the quality of reconstructed or compressed images relative to the original.

Limitations and caveats

While RMSE is widely used, practitioners should be aware of its limitations:

1. Scale dependence. RMSE is not comparable across datasets with different target scales unless normalized.

2. Outlier sensitivity. A small number of large errors can inflate RMSE substantially, potentially giving a misleading impression of overall model performance. In such cases, MAE or the median absolute deviation (MAD) may be more representative.

3. No directional information. RMSE does not distinguish between over-prediction and under-prediction because errors are squared. A model that systematically overestimates by 5 units and a model with random errors averaging 5 units in either direction will have similar RMSE values.

4. Ambiguity as an "average" error. Willmott and Matsuura (2005) pointed out that RMSE varies with the variability of the error distribution, not just the average error. MAE is a more straightforward measure of the average error magnitude.^[2]

5. Sensitivity to sample size in comparisons. When comparing RMSE values across studies, differences in sample size and data composition can affect the metric, making direct comparisons unreliable without careful normalization.^[11]

6. Not a complete model assessment. RMSE alone does not capture model calibration, uncertainty quantification, or the spatial/temporal distribution of errors. It should be used alongside other metrics and diagnostic plots (residual plots, Q-Q plots).

Historical context

The mathematical foundations of RMSE trace back to the development of the method of least squares. Adrien-Marie Legendre published the first clear exposition of least squares in 1805. Carl Friedrich Gauss claimed to have used the method since 1795, and in 1809 connected it to probability theory and the normal distribution. Within a decade of Legendre's publication, least squares had become the standard tool in astronomy and geodesy.^[7]

The modern use of RMSE as an evaluation metric grew out of this tradition. The squaring of residuals (central to both least squares and RMSE) is mathematically tied to the normal distribution through maximum likelihood theory. As statistical methods spread from the physical sciences into economics, psychology, engineering, and eventually machine learning, RMSE became one of the default measures of prediction accuracy.

Implementation examples

RMSE can be computed easily in all major programming languages and machine learning frameworks.

scikit-learn (Python)

from sklearn.metrics import root_mean_squared_error

y_true = [3, -0.5, 2, 7]
y_pred = [2.5, 0.0, 2, 8]

rmse = root_mean_squared_error(y_true, y_pred)
print(rmse)  # Output: 0.6123724356957945

In older versions of scikit-learn (before v1.4), RMSE was computed using mean_squared_error with squared=False:^[10]

from sklearn.metrics import mean_squared_error
import math

rmse = math.sqrt(mean_squared_error(y_true, y_pred))

PyTorch

import torch
import torch.nn as nn

y_true = torch.tensor([3.0, -0.5, 2.0, 7.0])
y_pred = torch.tensor([2.5, 0.0, 2.0, 8.0])

mse_loss = nn.MSELoss()
rmse = torch.sqrt(mse_loss(y_pred, y_true))
print(rmse.item())  # Output: 0.6123724356957945

NumPy (from scratch)

import numpy as np

def rmse(y_true, y_pred):
    return np.sqrt(np.mean((np.array(y_true) - np.array(y_pred)) ** 2))

print(rmse([3, -0.5, 2, 7], [2.5, 0.0, 2, 8]))  # Output: 0.6123724356957945

R

y_true <- c(3, -0.5, 2, 7)
y_pred <- c(2.5, 0.0, 2, 8)

rmse <- sqrt(mean((y_true - y_pred)^2))
print(rmse)  # Output: 0.6123724

Explain Like I'm 5 (ELI5)

Imagine you are trying to guess how many apples are in different baskets. After you guess, someone counts the actual number. Sometimes you guess too many, and sometimes you guess too few.

RMSE is a way to figure out how wrong your guesses are, on average. Here is what you do: take each guess and compare it to the real number. Figure out how far off you were. Then multiply each difference by itself (squaring it). Add up all those results, divide by how many baskets you guessed, and then take the square root of that number.

Why go through all this squaring and square-rooting? Because it makes big mistakes count extra. If you were off by 1 apple, the squared error is 1. If you were off by 5 apples, the squared error is 25. That is 25 times worse, not just 5 times worse. So RMSE punishes big blunders more than small slip-ups.

The final number you get is in apples (the same thing you were measuring), so it is easy to understand. If the RMSE is 2, it means your guesses are typically about 2 apples off from the real number. A smaller RMSE means better guessing.

References

1. Hastie, T., Tibshirani, R., & Friedman, J. (2009). *The Elements of Statistical Learning: Data Mining, Inference, and Prediction* (2nd ed.). Springer. ↩
2. Willmott, C. J., & Matsuura, K. (2005). "Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance." *Climate Research*, 30, 79-82. ↩
3. Chai, T., & Draxler, R. R. (2014). "Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding RMSE in the literature." *Geoscientific Model Development*, 7, 1247-1250. ↩
4. Hodson, T. O. (2022). "Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not." *Geoscientific Model Development*, 15(14), 5481-5487. ↩
5. Goodfellow, I., Bengio, Y., & Courville, A. (2016). *Deep Learning*. MIT Press. Section 5.5.1. ↩
6. Bishop, C. M. (2006). *Pattern Recognition and Machine Learning*. Springer. Section 1.2.5. ↩
7. Stigler, S. M. (1981). "Gauss and the Invention of Least Squares." *Annals of Statistics*, 9(3), 465-474. ↩
8. Hyndman, R. J., & Koehler, A. B. (2006). "Another look at measures of forecast accuracy." *International Journal of Forecasting*, 22(4), 679-688. ↩
9. James, G., Witten, D., Hastie, T., & Tibshirani, R. (2021). *An Introduction to Statistical Learning* (2nd ed.). Springer. ↩
10. scikit-learn documentation. "sklearn.metrics.root_mean_squared_error." https://scikit-learn.org/stable/modules/generated/sklearn.metrics.root_mean_squared_error.html ↩
11. Willmott, C. J., Robeson, S. M., & Matsuura, K. (2012). "A refined index of model performance." *International Journal of Climatology*, 32(13), 2088-2094. ↩
12. Murphy, A. H. (1988). "Skill Scores Based on the Mean Square Error and Their Relationships to the Correlation Coefficient." *Monthly Weather Review*, 116(12), 2417-2424. ↩
13. Nguyen, T., et al. (2023). "ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling." *arXiv:2307.01909*. ↩
14. Netflix Prize. "Grand Prize awarded to team BellKor's Pragmatic Chaos." Netflix Prize Forum / "Netflix Prize," Wikipedia. https://en.wikipedia.org/wiki/Netflix_Prize ↩