Small sample properties of forecasts from autoregressive models under structural breaks

Abstract

This paper develops a theoretical framework for the analysis of small-sample properties of forecasts from general autoregressive models under structural breaks. Finite-sample results for the mean squared forecast error of one-step ahead forecasts are derived, both conditionally and unconditionally, and numerical results for different types of break specifications are presented. It is established that forecast errors are unconditionally unbiased even in the presence of breaks in the autoregressive coefficients and/or error variances so long as the unconditional mean of the process remains unchanged. Insights from the theoretical analysis are demonstrated in Monte Carlo simulations and on a range of macroeconomic time series from G7 countries. The results are used to draw practical recommendations for the choice of estimation window when forecasting from autoregressive models subject to breaks.

Introduction

Autoregressive models are used extensively in forecasting throughout economics and finance and have proved so successful and difficult to outperform in practice that they are frequently used as benchmarks in forecast competitions. Due in large part to their relatively parsimonious form, autoregressive models are frequently found to produce smaller forecast errors than those associated with models that allow for more complicated nonlinear dynamics or additional predictor variables, c.f. Stock and Watson (1999) and Giacomini (2002).

Despite their empirical success, there is now mounting evidence that the parameters of autoregressive (AR) models fitted to many economic time series are unstable and subject to structural breaks. For example, Stock and Watson (1996) undertake a systematic study of a wide variety of economic time series and find that the majority of these are subject to structural breaks. Alogoskoufis and Smith (1991) and Garcia and Perron (1996) are other examples of studies that document instability related to the autoregressive terms in forecasting models. Clements and Hendry (1998) view structural instability as a key determinant of forecasting performance.

This suggests a need to study the behaviour of the parameter estimates of AR models as well as their forecasting performance when these models undergo breaks. Despite the interest in econometric models subject to structural breaks, little is known about the small sample properties of AR models that undergo discrete changes. In view of the widespread use of AR models in forecasting, this is clearly an important area to investigate. The presence of breaks makes the focus on small sample properties more relevant: even if the combined pre- and post-break sample is very large, the occurrence of a structural break means that the post-break sample will often be quite small so that asymptotic approximations may not be nearly as accurate as is normally the case.

A key question that arises in the presence of breaks is how much data to use to estimate the parameters of the forecasting model that minimizes a loss function such as root mean squared forecast error (RMSFE). We show that the RMSFE-minimizing estimation window crucially depends on the size of the break as well as its direction (i.e., does the break lead to higher or lower persistence) and which parameters it affects (i.e., the mean, variance or autoregressive slope parameters). In some situations the optimal estimation window trades off an increased bias introduced by using pre-break data against a reduction in forecast error variance resulting from using a longer window of the data. However, in other situations the small sample bias in the autoregressive coefficients may in fact be reduced after introducing pre-break data if the size of the break is small or even when the break is large provided that it is in the right direction (e.g., when persistence declines).

In the presence of parameter instability it is common to use a rolling window estimator that makes use of a fixed number of the most recent data points, although the size of the rolling window is based on pragmatic considerations rather than on an empirical analysis of the underlying time series process. Another possibility would be to test for breaks in the parameters and/or error variances and only use data after the most recent break, assuming a break is in fact detected. Alternatively, if no statistically significant break is found, an expanding window estimator could be used. Our theoretical analysis allows us to better understand when each of these procedures is likely to work well and why it is generally best to use pre-break data when forecasting using autoregressive models. First, breaks in the autoregressive parameters need not introduce bias in the forecasts (at least unconditionally). This tends to happen when an autoregressive coefficient declines after a break or the break only occurs in the intercept or variance parameter. Including pre-break data in such cases will tend to lead to a decline in RMSFE due to both a smaller squared bias and a reduction in the variance of the parameter estimate. Furthermore, in practice, there is likely to be a considerable error in detecting and estimating the point of the break of the autoregressive model. This leads to a worse performance of a post-break estimation procedure but also makes determination of the length of a rolling window more difficult.

Several practical recommendations emerge from our analysis regarding the choice of estimation window when forecasting from autoregressive models. First, for the macroeconomic data examined here, in general it appears to be difficult in practice to outperform expanding or long rolling window estimation methods. Unlike the case with exogenous regressors, forecasts from autoregressive models can be seriously biased even if only post-break observations are used. Including pre-break data in estimation of autoregressive models can simultaneously reduce the bias and the variance of the forecast errors. In most applications where breaks are not too large, expanding window methods or rolling window procedures with relatively large window sizes are likely to perform well. This conclusion may not of course carry over to longer data sets, e.g. high frequency financial data with thousands of observations, where estimation uncertainty can be reduced more effectively than with the relatively short macroeconomic data considered here.

The main contributions of this paper are as follows. First, we present a new procedure for computing the exact small sample properties of the parameters of AR models of arbitrary order, thus extending the existing literature that has focused on the AR(1) model. Our approach allows for fixed or random starting points and considers stationary AR models as well as models with unit root dynamics. We allow for the possibility of the AR model to switch from a unit root process to a stationary one and vice versa. Such regime switches could be particularly relevant to the analysis of inflation in a number of OECD countries since the first oil price shock in early 1970s. In addition to considering properties such as bias in the parameters, we also consider the RMSFE in finite samples. Second, we extend existing results on exact small sample properties of AR models to allow for a break in the underlying data generating process. We establish that one-step ahead forecast errors from AR models are unconditionally unbiased even in the presence of breaks in the autoregressive coefficients and in the error variances so long as the unconditional mean of the process remains unchanged. Our results also apply to models with unit roots. This extends Fuller's (1996) result obtained for AR models with fixed parameters, and generalizes a related finding due to Clements and Hendry (1999, pp. 39–42). Third, we present extensive numerical results quantifying the effect of the sizes of the pre-break and post-break data windows on parameter bias and RMSFE. Fourth, we undertake an empirical analysis for a range of macroeconomic time series from the G7 countries that compares the forecasting performance of expanding window, rolling window and post-break estimators. This analysis which allows for multiple breaks at unknown times confirms that, at least for macroeconomic time series such as those considered here, it is generally best to use pre-break data in estimation of the forecasting model.

The outline of the paper is as follows. Section 2 provides a brief overview of the small sample properties of the first-order autoregressive model that has been extensively studied in the extant literature. Theoretical results allowing us to characterize the small sample distribution of the parameters and forecast errors of autoregressive models are introduced in Section 3. Section 4 presents numerical results for AR models subject to breaks and Section 5 presents empirical results for a range of macroeconomic time series. Section 6 concludes with a summary and a discussion of possible extensions to our work.