Time Series Analysis

Time series analysis is the statistical study of ordered observations when dependence across time is part of the signal rather than a nuisance to ignore. Analysts use it to describe temporal structure, estimate dynamic relationships, detect changes, and forecast future observations with quantified uncertainty. Successive values may be correlated, variability may change over time, and the process generating the data may shift, so methods built for independent observations can give misleading standard errors, validation results, or forecasts.^[1][2]

The companion article on time series defines the data structure and surveys applications, model families, forecasting competitions, and recent pretrained models. This article instead covers methodology: how to frame a question, inspect and transform a series, specify a stochastic model, diagnose its residuals, validate forecasts, and distinguish predictive evidence from causal evidence. The two pages are an editorial division of one field, not descriptions of different kinds of time-indexed data.

Time series analysis is broader than forecasting. A descriptive analysis may estimate trend, seasonality, cycles, or dependence. An inferential analysis may test a parameter or compare dynamic relationships. An intervention analysis asks whether a known event is consistent with a level or slope change. A forecasting analysis estimates a future conditional distribution. Those goals can use related models, but they require different assumptions and validation designs.^[1][2]

Framing the analysis

The first decision is the target question, not the algorithm. An analyst should specify the measured variable, observational unit, time index, sampling interval, information available at each decision time, and output that will be judged. For forecasting, that also means fixing the forecast origin, horizon, update schedule, and loss or scoring rule. A one-step operational forecast, a twelve-month planning distribution, and an estimate of an intervention effect are not interchangeable targets.^[2]

The measurement process belongs in the model specification. Timestamps can refer to when an event occurred, when it was recorded, or when it became available to a decision maker. Published economic series may be revised. Sensor values may be averaged before storage. Transaction counts may use a local-day boundary that changes at daylight-saving transitions. If the analysis reconstructs information that was not actually available at a historical forecast origin, the backtest contains look-ahead bias even when every row is in chronological order.

A useful analysis plan records four elements before model fitting:

Scroll sideways for more →

This separation prevents an accurate fit to the wrong target from being treated as a successful analysis.

Inspection, sampling, and transformations

Plot the observations in time order before reducing them to summary statistics. A full time plot can reveal level shifts, changing variance, calendar effects, gaps, recording limits, and individual observations that deserve investigation. Seasonal subseries plots and plots grouped by hour, weekday, month, or another known cycle can show whether a seasonal pattern is stable or changes across years. Lag plots and autocorrelation plots then describe dependence that is not visible in a histogram.^[1]

Standard autocorrelation calculations assume equally spaced observations: lag 1 means one sampling interval, not merely the previous recorded event. Irregularly sampled clinical records, event logs, and sensor streams therefore require an explicit treatment of elapsed time. Resampling and interpolation can make regular-grid methods available, but they create modeled values and can suppress short-lived behavior. Continuous-time models, state-space formulations, or architectures that include observation times are alternatives. Missingness indicators may be needed when whether or when a measurement was taken carries information.^[3]

Transformations should answer a diagnosed problem. A logarithm or a Box-Cox power transformation can make multiplicative variation more nearly additive or stabilize variance, but it also changes the scale on which errors and forecasts are interpreted.^[4] The transformation parameter is conventionally written as lambda. Back-transforming a conditional mean on the transformed scale does not generally produce the conditional mean on the original scale, so point forecasts and intervals need the appropriate transformation adjustment.

Differencing replaces a level by a change. A first difference is y(t) - y(t - 1); a seasonal difference at period m is y(t) - y(t - m). Differencing can remove a stochastic trend or recurring seasonal level, but unnecessary differencing adds dependence and discards level information. Deterministic trend terms, regression effects, decomposition, and stochastic differencing represent different assumptions and should not be selected only because one produces a visually flatter line.

Decomposition separates an observed series into estimated components, commonly trend-cycle, seasonal, and remainder. STL uses repeated locally weighted regressions to estimate an additive decomposition, allows the amount of trend and seasonal smoothing to be varied, and includes a robust option that reduces the influence of unusual observations.^[5] A decomposition is a descriptive filter, not proof that the remainder is independent noise. The remainder still needs diagnostic checking if it will be used as the error process of a model.

Stationarity and temporal dependence

For a weakly stationary process, the mean is constant and the covariance between two observations depends on their lag rather than their calendar dates. A finite sample cannot prove that a process has this property. Trend, changing seasonal behavior, variance shifts, and structural breaks can all make a stationary specification implausible over the observed span.

Unit-root and stationarity tests ask different null questions. The Dickey-Fuller family tests a unit-root null in an autoregressive specification.^[6] The KPSS test reverses the direction: its null is level or trend stationarity, depending on the chosen version, against a unit-root alternative.^[7] Failure to reject either null is not confirmation that the null is true, and disagreement between the tests does not choose a transformation automatically. Lag length, deterministic terms, sample size, and breaks affect the result. Perron's analysis showed that allowing a one-time change in level or trend can reverse unit-root conclusions for important macroeconomic series.^[8]

The sample autocorrelation function, or ACF, estimates correlation between values separated by each lag. The partial autocorrelation function, or PACF, estimates the association at a lag after accounting for shorter lags. Both are exploratory estimates with sampling uncertainty. The NIST handbook notes that the usual ACF formula assumes equal spacing and uses plots across many lags for model identification.^[9] Peaks at a known period can indicate seasonality, while slow decay can be consistent with nonstationarity, but a plot alone does not identify a unique model.

Two meanings of "moving average" must be kept separate. A moving-average smoother replaces observations by local averages. A stochastic MA(q) model instead expresses the current observation as a linear combination of current and lagged innovations. Confusing the two leads to incorrect descriptions of ARIMA, whose MA term is the stochastic model component rather than a rolling mean.

Model representations

No model family is best independently of the target, sampling process, available covariates, and validation results. The main statistical representations answer different structural questions.

ARIMA and seasonal models

An ARIMA(p,d,q) model applies d differences and models the resulting series with p autoregressive terms and q moving-average innovation terms. Seasonal ARIMA adds analogous terms at a specified seasonal period. The autoregressive terms condition on past observations; the moving-average terms represent serial structure in past forecast errors. The orders are model dimensions to identify and validate, not generic hyperparameters whose best values transfer between data sets.^[1][9]

Regression terms can be combined with ARIMA errors for calendar effects, interventions, prices, weather, or other predictors. Future values of a predictor must actually be known or separately forecast at prediction time. Treating realized future weather or demand drivers as if they were known creates a favorable but unusable backtest.

State-space models

A state-space model separates an unobserved state transition from the observation equation. This representation can express local levels, stochastic trends, seasonal components, dynamic regression coefficients, and missing observations in one probabilistic system. The Kalman filter recursively updates a linear state estimate and its uncertainty as each observation arrives; Kalman's original formulation also covered nonstationary statistics.^[10] Smoothing estimates earlier states using later observations, which is useful for retrospective analysis but must not be mistaken for an online estimate available at the earlier date.

Multivariate and long-run relationships

A vector autoregression, or VAR, models each variable as a function of lags of all variables in the system. It provides a flexible reduced-form description of multivariate dynamics, but the estimated innovations are not automatically structural shocks. Sims's influential formulation used VARs to challenge strong identifying restrictions in macroeconomic models, while later structural interpretations require additional assumptions.^[11]

Regressing unrelated nonstationary series on each other can produce apparently strong relationships. Granger and Newbold demonstrated this spurious-regression problem with simulated time series.^[12] When nonstationary variables share a stationary linear combination, they are cointegrated; an error-correction representation can combine short-run changes with adjustment toward that long-run relation.^[13] Tests for unit roots, cointegration rank, lag order, and parameter stability are therefore part of the specification rather than optional decorations.

Granger predictability asks whether the past of one variable improves prediction of another conditional on the information set in the model.^[14] It does not by itself establish a manipulable causal effect. Omitted variables, aggregation, sampling rate, anticipation, and incorrect lag structure can change the conclusion.

Frequency-domain methods

Time-domain models describe dependence by lags. Frequency-domain analysis represents variance and co-movement by frequency, using tools such as the periodogram, spectral density, cross-spectrum, coherence, and phase. It is useful when periodic or oscillatory behavior is central. Frequency peaks still require uncertainty assessment and substantive interpretation; a peak can arise from filtering, finite samples, changing periodicity, or the observation process rather than a stable physical cycle.

Identification, estimation, and diagnostics

Model identification combines subject knowledge, plots, transformations, candidate structures, and a comparison criterion. Maximum likelihood estimates parameters under a specified probability model; conditional or exact likelihood conventions can produce different values for short series. Least squares may estimate some dynamic regressions, but serially correlated errors invalidate independent-error standard errors unless the dependence is modeled or the inference is adjusted.

Information criteria compare fit with a penalty for model complexity. Akaike's criterion was developed as an information-based method for statistical model identification and, in its familiar likelihood form, adds twice the number of fitted parameters to negative twice the maximized log likelihood.^[15] Schwarz's criterion, often called BIC, replaces that constant per-parameter penalty with one proportional to the logarithm of sample size.^[16] Their numerical values are relative, not absolute measures of adequacy. Candidates should use the same response data and compatible likelihood definitions, and the selected model still needs residual and out-of-sample checks.

A residual is the observed value minus its fitted or forecast value under a stated information set. Useful diagnostics include:

• a residual time plot for remaining level shifts, variance changes, and isolated errors;
• a residual ACF for remaining serial correlation;
• residuals against fitted values and relevant covariates for misspecified shape or variance;
• distributional checks when a likelihood or prediction interval relies on a particular innovation distribution;
• stability checks across time, season, and forecast horizon.

The Ljung-Box statistic combines residual autocorrelations across a chosen set of lags to test lack of fit in autoregressive-moving-average models.^[17] A small p-value is evidence that serial structure remains under the test specification. A large p-value is not proof of independence, and the result depends on the lag set, estimated degrees of freedom, and sample size. An ACF with no prominent spikes can also miss nonlinear dependence.

Uncorrelated residuals can still have changing conditional variance. Engle's ARCH model represented current innovation variance as a function of past squared innovations, making volatility clustering a modeled feature rather than treating it as a constant-error process.^[18] ACFs or portmanteau tests on squared residuals can reveal this kind of remaining structure. Whether it matters depends on the goal: it may have little effect on a point conditional mean but materially affect intervals, risk estimates, and likelihood-based inference.

Diagnostics are model criticism, not a ritual for certifying truth. If several models pass the same residual tests, validation and domain plausibility can distinguish them. If every candidate fails in the same period, the problem may be a missing intervention, changed measurement process, structural break, or wrong sampling resolution rather than an insufficiently large lag order.

Time-aware validation and forecast evaluation

Forecast evaluation must reproduce the information boundary at each historical origin. Rolling-origin evaluation fits or updates a model using observations available up to an origin, forecasts the next observation or block, advances the origin, and aggregates errors across origins. The design can use an expanding training window or a fixed-length rolling window. Multiple origins reduce dependence on one fortunate test interval and expose performance across regimes.^[2]

The horizon and update policy must match deployment. One-step validation does not establish multi-step accuracy. A model refit after every observation is not directly comparable with a production model updated monthly. Scaling, imputation, decomposition, lag creation, feature engineering, and model selection must be learned within each training window. Computing them once on the complete series leaks later information even if the final estimator sees only past target values.

Random k-fold cross-validation is not universally invalid for time series. Bergmeir, Hyndman, and Koo showed that it can be valid for purely autoregressive prediction when the fitted models have uncorrelated errors.^[19] That result is narrower than a license to shuffle any temporal data. Rolling-origin evaluation remains the direct choice when order, nonstationarity, data availability, or a multi-step operational horizon matters.

Accuracy measures encode different losses:

Scroll sideways for more →

Hyndman and Koehler documented degeneracies in common percentage measures and proposed MASE for comparison across series.^[20] No single metric is a universal default. The chosen measure should reflect decision costs, be reported with a baseline, and be summarized across origins without hiding important horizons or subgroups.

A point forecast omits uncertainty. Quantile, interval, and full-distribution forecasts should be assessed with proper scoring rules that reward accurate and honest predictive distributions.^[21] Calibration asks whether outcomes are statistically consistent with the forecast probabilities; sharpness asks how concentrated the forecasts are. Sharpness is useful only subject to calibration, because an arbitrarily wide interval can achieve high empirical coverage while conveying little information.^[22] Coverage should therefore be reported with interval width or a proper score and examined by horizon and regime.

Interventions, causality, and structural change

Temporal order helps rule out some explanations, but it does not convert association into causation. A predictive relationship may reflect a common driver, feedback, measurement timing, or an omitted variable. Granger predictability is explicitly relative to the variables and lags in the information set.^[14] Strong causal claims require a design and assumptions that identify the counterfactual outcome, not only a significant lag coefficient.

Intervention analysis represents a known event with a specified response shape while modeling the dependent noise. Box and Tiao used difference-equation models to estimate level changes and dynamic intervention effects in economic and environmental series.^[23] Credible interpretation depends on a defensible intervention time, sufficient pre-intervention structure, no unmodeled concurrent change, and an error model that captures serial dependence. Trying many event dates or response shapes and reporting only the strongest one invalidates ordinary significance statements unless that search is included in the analysis.

Change-point methods instead search for unknown boundaries where a distributional parameter changes. PELT, for example, uses dynamic programming with pruning to optimize a penalized segmentation and can have computational cost close to linear under stated conditions.^[24] Detecting a change point does not identify its cause. It marks a location at which a chosen statistical model changes enough to outweigh its penalty.

Structural change also limits forecast validity. A model can fit historical residuals well and fail after a policy, product, sensor, market, or user population changes. This is a form of distribution shift. Evaluation across multiple origins, explicit break analysis, and monitoring after deployment are complementary: none guarantees that the next regime will resemble the past.

Reproducible workflow

A defensible time series analysis leaves enough information to reconstruct what was known and what was estimated:

1. Define the target, clock, sampling rule, horizon, update schedule, and decision loss.
2. Preserve raw timestamps and revisions; document aggregation, exclusions, gaps, and time-zone handling.
3. Plot the series and inspect seasonal, lagged, and missingness structure before transformation.
4. Record every transformation and whether it was estimated inside each training window.
5. Compare interpretable baselines with candidate models selected for the stated question.
6. Inspect residual level, dependence, variance, distribution, and stability, not only in-sample fit.
7. Evaluate at multiple historical origins and at the operational horizons.
8. Report point error, probabilistic calibration or scores, uncertainty, and important subgroup failures.
9. Separate predictive, intervention, and causal conclusions.
10. Monitor data definitions, residuals, calibration, and performance after deployment.

The objective is not to find a method that makes every diagnostic insignificant. It is to make the assumptions, information boundary, comparison, and remaining uncertainty visible enough that another analyst can challenge or reproduce the conclusion.

References

1. ^NIST/SEMATECH, "Introduction to Time Series Analysis," e-Handbook of Statistical Methods. itl.nist.gov/...pmc4
2. ^Tashman, L. J. "Out-of-sample tests of forecasting accuracy: an analysis and review." International Journal of Forecasting 16(4), 437-450 (2000). doi.org/...S0169-2070(00)00065-0
3. ^Shukla, S. N., Marlin, B. M. "A Survey on Principles, Models and Methods for Learning from Irregularly Sampled Time Series." arXiv:2012.00168 (2020, revised 2021). arxiv.org/...2012.00168
4. ^Box, G. E. P., Cox, D. R. "An Analysis of Transformations." Journal of the Royal Statistical Society Series B 26(2), 211-243 (1964). doi.org/...j.2517-6161.1964.tb00553.x
5. ^Cleveland, R. B., Cleveland, W. S., McRae, J. E., Terpenning, I. "STL: A Seasonal-Trend Decomposition Procedure Based on Loess." Journal of Official Statistics 6(1), 3-73 (1990). proquest.com/...1266805989
6. ^Dickey, D. A., Fuller, W. A. "Distribution of the Estimators for Autoregressive Time Series with a Unit Root." Journal of the American Statistical Association 74(366a), 427-431 (1979). doi.org/...01621459.1979.10482531
7. ^Kwiatkowski, D., Phillips, P. C. B., Schmidt, P., Shin, Y. "Testing the null hypothesis of stationarity against the alternative of a unit root." Journal of Econometrics 54(1-3), 159-178 (1992). doi.org/...0304-4076(92)90104-Y
8. ^Perron, P. "The Great Crash, the Oil Price Shock, and the Unit Root Hypothesis." Econometrica 57(6), 1361-1401 (1989). doi.org/...1913712
9. ^NIST/SEMATECH, "Autocorrelation," e-Handbook of Statistical Methods. itl.nist.gov/...eda35c
10. ^Kalman, R. E. "A New Approach to Linear Filtering and Prediction Problems." Journal of Basic Engineering 82(1), 35-45 (1960). doi.org/...1.3662552
11. ^Sims, C. A. "Macroeconomics and Reality." Econometrica 48(1), 1-48 (1980). doi.org/...1912017
12. ^Granger, C. W. J., Newbold, P. "Spurious regressions in econometrics." Journal of Econometrics 2(2), 111-120 (1974). doi.org/...0304-4076(74)90034-7
13. ^Engle, R. F., Granger, C. W. J. "Co-Integration and Error Correction: Representation, Estimation, and Testing." Econometrica 55(2), 251-276 (1987). doi.org/...1913236
14. ^Granger, C. W. J. "Investigating Causal Relations by Econometric Models and Cross-spectral Methods." Econometrica 37(3), 424-438 (1969). doi.org/...1912791
15. ^Akaike, H. "A New Look at the Statistical Model Identification." IEEE Transactions on Automatic Control 19(6), 716-723 (1974). doi.org/...TAC.1974.1100705
16. ^Schwarz, G. "Estimating the Dimension of a Model." The Annals of Statistics 6(2), 461-464 (1978). doi.org/...1176344136
17. ^Ljung, G. M., Box, G. E. P. "On a measure of lack of fit in time series models." Biometrika 65(2), 297-303 (1978). doi.org/...65.2.297
18. ^Engle, R. F. "Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation." Econometrica 50(4), 987-1007 (1982). doi.org/...1912773
19. ^Bergmeir, C., Hyndman, R. J., Koo, B. "A note on the validity of cross-validation for evaluating autoregressive time series prediction." Computational Statistics & Data Analysis 120, 70-83 (2018). doi.org/...j.csda.2017.11.003
20. ^Hyndman, R. J., Koehler, A. B. "Another look at measures of forecast accuracy." International Journal of Forecasting 22(4), 679-688 (2006). doi.org/...j.ijforecast.2006.03.001
21. ^Gneiting, T., Raftery, A. E. "Strictly Proper Scoring Rules, Prediction, and Estimation." Journal of the American Statistical Association 102(477), 359-378 (2007). doi.org/...016214506000001437
22. ^Gneiting, T., Balabdaoui, F., Raftery, A. E. "Probabilistic forecasts, calibration and sharpness." Journal of the Royal Statistical Society Series B 69(2), 243-268 (2007). doi.org/...j.1467-9868.2007.00587.x
23. ^Box, G. E. P., Tiao, G. C. "Intervention Analysis with Applications to Economic and Environmental Problems." Journal of the American Statistical Association 70(349), 70-79 (1975). doi.org/...01621459.1975.10480264
24. ^Killick, R., Fearnhead, P., Eckley, I. A. "Optimal Detection of Changepoints With a Linear Computational Cost." Journal of the American Statistical Association 107(500), 1590-1598 (2012). doi.org/...01621459.2012.737745