Time Series Models

Time-series is a sequence of measurements of a variable, approximately equally spaced in time. The term is typically used on economic and financial data.

Frequency of time-series: yearly, quarterly (public company report, national product), monthly (government statistics); weekly, daily (gas price, stock index); per hour (weather, tide height), minute, second or higher frequencies (equity trading);

Characteristics of time-series: trend (drift in moving average); seasonality (spectral density estimation); autocorrelation (dependency on previous state, e.g. random walk); stationarity (time-invariant distribution);

Uncertainty in time-series models are often presented as 95% (also 80%) prediction interval band, as in {Hanna2017}.

Decomposition Models

Additive and multiplicative decomposition, which are identical under log transformation:

\[ X = T + C + S + I \] \[ X = T * C * S * I \]

Components of time-series:

1. Trend (T): non-periodic low-frequency evolution;
2. Seasonal (S): periodic fluctuations, observable frequencies depend on time-series frequency;
   1. "Cycle": multi-year periods attributed to the "business cycle", often estimated together with trend as the "trend-cycle" component;
   2. Annual pattern: solar season (temperature, precipitation, length of day);
   3. Weekly pattern: day of week;
   4. Daily pattern: solar time, time zone (including daylight saving time);
3. Irregular (I): residual after removing model components;
4. Calendar effects: any effect related to the calendar and its changes;
   • Fixed holidays (national days, Thanksgiving, Christmas - New Year);
   • Moving holiday effects are caused by holidays whose dates vary from year to year, such as U.S. Monday holidays and holidays using non-Gregorian calendars (Easter, Rosh Hashanah, Eid al-Fitr, Chinese New Year, Diwali);
   • For monthly time-series, trading day (TD) effects are caused by changes in the number of business days in each month;
5. Outliers are abrupt, atypical movements in the time series, caused by extreme events:
   • severe weather (hurricane, storm in summer/fall; blizzard in winter);
   • social unrest (strike);

Causes of seasonal fluctuations in economic time-series include calendar (business day, holiday), timing (school vacation, payday, tax/accounting period), weather, and social expectation of seasonal patterns {Granger1979}. Different time-series can have different causes of seasonal, and thus shall be modeled differently.

Seasonal adjustment is the removal of seasonal components from a time-series. This is done to reveal the low frequency components (commonly known as "trend-cycle") of economic time-series, which are usually both statistically and economically important. It also removes one possible source of spurious relationship among multiple time-series.

Classical decomposition (stats::decompose()) uses moving average filter for trend component, and assumes seasonal component is periodic; its estimate is not available at end points, and not robust to outliers.

X-12-ARIMA and TRAMO-SEATS

X-13ARIMA-SEATS is a seasonal adjustment software combining X-12-ARIMA (developed by the United States Census Bureau) and TRAMO-SEATS (developed by the Band of Spain). This seasonal adjustment software is currently used by the US Census Bureau, and also by many government agencies around the world.

X-12-ARIMA decomposition (seasonal::seas()) for quarterly and monthly time-series refines the classical decomposition by iteration; its estimate is available at end points and relatively robust to outliers; the seasonal component can vary slowly over time {Ladiray2001}. Also for quarterly and monthly time-series, SEATS decomposition {Dagum2016}.

The X-11 method decomposes a time-series into trend-cycle, seasonal, and irregular components by iteratively applying linear filters (moving averages).

Time Series Regression with ARIMA noise (TRAMO); Seasonal Extraction in ARIMA Time Series (SEATS);

To protect seasonal effect estimates from distortion by outliers, generic outlier regressors can be used to estimate and temporarily remove the outliers.

Local Regression

Seasonal and trend decomposition using LOESS, aka STL decomposition (stats::stl()), can handle any type of seasonality, with variable seasonal component and smoothness of the trend-cycle component, robust to outliers {Cleveland1990}.

Exponential Smoothing

exponential smoothing methods weights exponentially decaying in observation age. forecast::ets() {Hyndman2008}

Multiple seasonality time-series

Dynamic harmonic regression with multiple seasonal periods;

Trigonometric Box–Cox transform, ARMA errors, Trend, and Seasonal components (TBATS) models {DeLivera2011};

Autoregressive Models

autocorrelation (ACF), partial autocorrelation (PACF) serial correlation lag

auto-regression (AR) vector auto-regression (VAR), implemented in R package vars.

Auto-regressive conditional heteroskedasticity (ARCH)

Moving average (MA) of order \(2k+1\): \[ \hat{T}_{t} = \frac{1}{2k+1} \sum_{j=-k}^k y_{t+j} \] \(m\)-MA means a moving average of order \(m\), and \(n \times m\)-MA means \(m\)-MA followed by \(n\)-MA. More sophisticated weights (kernel()) can be used for smoother results.

ARIMA

AutoRegressive Integrated Moving Average (ARIMA) is a versatile family of models for modeling and forecasting time series data. (Box-Jenkins) stats::arima(), forecast::Arima() {Hyndman2008}.

Seasonal ARIMA models have a special form for efficiently modeling many kinds of seasonal time series and are heavily used in seasonal adjustment.

Regression+ARIMA models use linear regression to estimate moving holiday, trading day and outlier effects, and then use a seasonal ARIMA model to estimate trend, cycle and seasonal components from the regression residuals.

Neural Network

Generic Bayesian neural network (BNN) model with Long short term memory (LSTM) encoder-decoder layers for large numbers of time-series provides better forecast accuracy at extreme events, as deployed in Uber {Laptev2017, Zhu2017}.