Does disagreement among oil price forecasters reflect future volatility? Evidence from the ECB Surveys Tarek Atallah,1 Fred Joutz2 and Axel Pierru3 Abstract For short forecast horizons, we find statistical evidence that the oil price volatility observed ex post explains ex-ante disagreement between oil price forecasters of the ECB’s professional survey. Since the forecasts considered are quarterly average prices, the observed disagreement is however not directly comparable to oil price volatility. We therefore use the geometric Brownian motion as a benchmark model to translate the observed disagreement into volatility. When applied to the ECB surveys, this method results in a disagreement-based volatility that is well correlated with the volatility observed ex post. These empirical findings are robust to the number of respondents considered. 1. Introduction When disagreement between oil-price forecasters increases, does this mean the future oil price has become more uncertain? We empirically address this issue using crude oil price forecasts from the European Central Bank’s Survey of Professional Forecasters (ECB SPF). In addition, this paper suggests a simple method to measure the oil price volatility consistent with the level of disagreement over the forecasted average price. Disagreement among forecasters is usually measured by the dispersion of point forecasts across the panel of respondents. Various reasons may be invoked to explain why oil price forecasters disagree. These forecasters may have different knowledge at the time the forecast is made, or produce their forecasts at different dates. They might disagree about the relevant exogenous variables or the way these variables translate into a specific price level. Disagreement might also result from the strategic behavior of certain forecasters, for instance to influence the oil market or to gain attention from the media. Lamont (2002) hypothesizes that if forecasters are paid according to relative ability, they might scatter, since it is hard to win when making a forecast similar to others. However, the uncertainty surrounding the oil price may significantly contribute to explain the disagreement observed between forecasters. As emphasized by Bowles et al. (2007), using disagreement as a measure of uncertainty makes sense, to the extent that different forecasters have a different assessment of the outlook, which, in turn, reflects the overall uncertainty surrounding the outlook. Another argument could be that the disparity in forecasters’ models and beliefs may lead to more divergent forecasts when oil price volatility is greater. A more volatile oil price would then lead to a higher disagreement among forecasters. The macroeconomic literature4 has already investigated possible explanations for disagreement among forecasters. A special attention5 has been given to the relationship between disagreement 1 King Abdullah Petroleum Studies and Research Center (KAPSARC). Scientific Visitor at KAPSARC & Department of Economics at The George Washington University. 3 King Abdullah Petroleum Studies and Research Center (KAPSARC). 2 1 and the uncertainty surrounding forecasted variables. However, most of these studies attempt to correlate measures of forecasts dispersion with forecasts errors and proxies of macroeconomic uncertainty. Surprisingly enough, little attention6 has so far been paid to the empirical analysis of disagreement between oil price forecasters, whereas the price volatility, either implied or realized, is a straightforward available measure of the uncertainty surrounding the oil price. Thus, using monthly oil price forecasts reported by Consensus Economics, Singleton (2012) finds that higher dispersion of forecasts is positively correlated with future increases in futures price volatility. The empirical analysis presented in this paper is based on ECB SPF oil price forecasts that, until now, had been used in two7 other studies only. Since these point forecasts relate to quarterly average prices, the observed disagreement cannot be directly compared to the oil price volatility. The distribution of the forecasts can however be interpreted as the distribution of the average price over the quarter considered. This raises the following question: how to infer an oil price volatility measure that is consistent with this distribution? Under the standard assumption that the oil price follows a geometric Brownian motion, we suggest a formula that serves to derive a price volatility from the distribution of forecasts. In other words, we use this simple reducedform model as a benchmark to translate the observed disagreement into volatility. When applied to the ECB surveys, this method results in a disagreement-based volatility that is well correlated with the volatility observed ex post. The next section presents the ECB SPF oil price forecasts. A measurement of disagreement among forecasters is then introduced. Section 3 investigates to which extent the oil price volatility observed ex post may explain our disagreement index. We then derive a disagreementbased volatility formula. Section 5 applies this formula to the ECB SPF disagreement index and uses regressions to see how well the resulting volatility fits the historical volatility. The last section concludes. 2. ECB SPF oil price forecasts The European Central Bank (ECB) has collected quarterly assumptions/forecasts of Brent crude oil prices since 2002q1 in its’ Survey of Professional Forecasters8 (SPF). These oil-price forecasts refer to the average nominal spot price of Brent over the quarter. The survey includes participants from the financial sector (mostly banks), non-financial research institutes and employer or employee organizations. The replies to the SPF are typically sent9 between days 16 and 21 of January (Q1 survey), April (Q2), July (Q3) and October (Q4). Thus, the survey participants have market information available to them for the first 15 days of each quarter. Our 4 For instance, Dopke and Fritsche (2006) and Siklos (2013) try to identify variables that could affect disagreement over inflation forecasts. 5 See for instance Bowles et al. (2007), 6 Including the studies on herding or anti-herding of oil price forecasters, like Pierdzioch et al. (2010). 7 Pierdzioch et al. (2010) analyze whether oil price forecasters herd or anti-herd. Reitz et al. (2012) investigate whether regressive and extrapolative expectations exhibit significant nonlinear dynamics. 8 The SPF collects point and probability estimates for Euro area annual HICP inflation, annual GDP growth, and the unemployment rate. In addition, they ask the participants to provide the assumptions they are using for the ECB’s interest rate for main refinancing operations, the crude oil price, the USD/EUR exchange rate, and the annual change compensation cost per employee or labor costs. 9 Communication with Victor Lopez Perez at the ECB-SPF on December 3rd, 2012. 2 sample period is from 2002q1-2012q4 which includes 44 survey rounds. Initially, the SPF surveyed forecasters for the current quarter and the subsequent next four quarters. We will refer to them as the 0-4 horizon forecasts. After 2010q1, the ECB stopped collecting the 4-quarterahead forecast. Figure 1 illustrates the time series of available horizon-0 forecasts for each quarter (see Figure A1 for the other horizons). In the first year, there were about 35-40 participants, thereafter participation fluctuates between 45 and 55. Since the scope for disagreement may be related to the number of respondents, we need to check the robustness of our analysis to the size of the group of forecasters considered. The level of participation by individual forecasters varies as some enter later than 2002q1, leave permanently at some quarter, and do not contribute to quarters consecutively, which produces an unbalanced panel. After examining the participation by the individual forecasters, we therefore define five arbitrary groups of forecasters, based on the minimum number of forecasts provided by each forecaster. Table A1 gives, for each horizon, the number of participants who provided a minimum number of forecasts: at least one forecast: 90 forecasters, 10 or more: 71 forecasters, 20 or more: 55 forecasters, 30 or more: 35 forecasters, 37 or more: 24 forecasters. The smaller numbers in horizon H4 are due to the discontinuation of collecting those forecasts after 2010Q1. Figure 1 : Number of available forecasts per quarter (horizon 0, 90 forecasters) For the sake of illustration, Figure 2 shows the mean forecasts of the group of 90 forecasters and the quarterly average nominal Brent price10. 10 Calculated as the average of daily closing prices (source: EIA). 3 Figure 2: Actual average price and ECB SPF mean forecast (all horizons, 90 forecasters) 3. Dispersion of forecasts, disagreement index and forecast uncertainty Let be the number of forecasts made in quarter t for horizon h that are considered. The uth price forecast is denoted as deviation of forecasts . The dispersion of forecasts is captured by the standard , with: (1) where is the mean forecast value Our measure11 of disagreement . in quarter t for horizon h is the ratio of the standard deviation of forecasts to the mean forecast: (2) 11 See for instance Siklos (2012) for alternative measures of forecast disagreement; note that the forecasters of the ECB professional survey only provide point estimates for the oil price, with no information on the underlying probability distributions. 4 In price , the forecast error is the difference between the actual average nominal Brent oil and the mean forecast made h quarters before: (3) If disagreement reflects forecast uncertainty, one would expect a positive correlation between the dispersion of forecasts and the subsequent forecast error. For horizon 0, the existence of such a correlation is suggested by Figure 3 that plots the absolute forecast error versus the standard deviation of forecasts when considering all available forecasts. Figure 3: Absolute forecast error versus standard deviation of forecasts (90 forecasters, horizon 0) Consequently, as a first approach12 to testing if disagreement may be linked to uncertainty, we regress the absolute value of the forecast error against the dispersion of forecasts. For every horizon and group of forecasters, we therefore use regressions of the type: (4) For all regressions performed in this paper, we apply the same following procedure. First, we check the stationary of both endogenous and exogenous variables with the ADF test. If the null assumption of a unit root cannot be rejected at a 5% level of significance, we perform the regression on the differenced series. Second, we do not consider regressions that yield residuals for which the null hypothesis of a normal distribution is rejected by the Jarque-Bera test at a 10% 12 By applying a similar approach to the dispersion of growth and inflation forecasts from a panel of German professional forecasters, Dopke and Fritsche (2006) do not find statistical evidence that dispersion is a reasonable measure of forecast uncertainty. 5 level of significance. Third, since residuals of most of the regressions performed are autocorrelated, we use the Newey-West variance estimator to achieve hypothesis testing. Fourth, only the regressions in which the slope coefficient is significant are reported in the paper. Table 1 gives the results of the regressions of type (). All slope coefficients relate to horizon 0 and are positive, which suggests that, in the short run, disagreement and forecast uncertainty are linked. Group of forecasters Horizon Constant Slope R2 90 71 0 0 -1.52 -1.23 1.59** 0.32 1.53** 0.31 55 24 0 0 -0.31 -0.46 1.32* 0.21 1.54** 0.30 Note: ***, **, and * denote statistical significance at 1%, 5%, and 10% levels, respectively; t-tests are based on standard errors corrected using the Newey-West procedure. Table 1: Regressions of absolute forecast error versus standard deviation of forecasts The next section studies the relationship between disagreement and the oil price volatility observed ex post, which is the main focus of this paper. 4. Does ex-post realized volatility explain ex-ante disagreement? First, we estimate the (realized) volatility observed ex-post. To do so, we use weekly prices, by considering the closing spot price of the last working day of every week in the quarter. The forecasts are assumed to be conditional upon all available information at the time the forecast is produced. Since we do not know the actual date of production of these forecasts, we consider two alternative assumptions: either the forecast is produced at the start of the quarter, or it is produced just before returning the questionnaire to the ECB. Let us consider any quarter t and let be the number of weekly prices observed from the start of quarter t until the end of the forecasted quarter at horizon h. We estimate the following two series of realized volatility, both computed as the standard deviation13 of price returns: - the ‘full quarter’ volatility corresponding to the assumption that the forecasts are produced when the first weekly price of the quarter is observed: 13 See for instance Sadorsky (2006) and Matar et al. (2013); we here do not adjust returns for convenience yield. When the price is assumed to follow a geometric Brownian motion as in Section 5, its volatility has to be estimated as the standard deviation of price returns. 6 (5) Where is the kth weekly price observed during the period considered. - the ‘deadline adjusted’ volatility corresponding to the assumption that the forecasts are produced just before the deadline to return the filled questionnaire to the ECB: (6) Where d is the number of weekly prices realized before the deadline14 to return the filled questionnaire to the ECB, as illustrated in Figure 4. Start of quarter t 1 Deadline to return filled questionnaire End of forecasted quarter at horizon h d N(h) wt,h vt,h Figure 4: Full quarter volatility and deadline adjusted volatility 14 Typically between the 17th and the 24th day of the first month; for each quarter, d is the last weekly price prior to the deadline indicated by the ECB at: http://www.ecb.int/stats/prices/indic/forecast/shared/files/SPF_rounds_dates.pdf?06a8d73c8231cca300071f251923c 9b9 7 Figure 5: ex-post volatilities and disagreement index, group of 90 forecasters Figure 6: ex-post volatilities and disagreement index, group of 24 forecasters 8 To test for the existence of a relationship between disagreement and price volatility, for every horizon and group of forecasters we perform regressions of both following types: (7) (8) Horizon Endogenous variable Exogenous variable Constant Slope R2 0 Disagreement index, 90 forecasters Full quarter volatility 0.02** 1.05*** 0.62 0 Disagreement index, 71 forecasters Full quarter volatility 0.02** 0.99*** 0.58 0 Disagreement index, 55 forecasters Full quarter volatility 0.02*** 0.85*** 0.51 0 Disagreement index, 35 forecasters Full quarter volatility 0.02*** 0.88*** 0.48 0 Disagreement index, 24 forecasters Full quarter volatility 0.01 0.92*** 0.47 0 Disagreement index, 24 forecasters 0 1 1 1 2 2 2 2 2 2 2 Deadline adjusted volatility 0.02** 0.80*** 0.40 Disagreement index, 55 forecasters ++ Deadline adjusted volatility --- 0.46*** 0.19 Disagreement index, 90 forecasters ++ Full quarter volatility --- 1.07*** 0.31 Disagreement index, 71 forecasters ++ Full quarter volatility --- 1.12*** 0.36 Disagreement index, 90 forecasters ++ Deadline adjusted volatility --- 0.88*** 0.32 Disagreement index, 90 forecasters ++ Full quarter volatility --- 1.24*** 0.18 Disagreement index, 71 forecasters ++ Full quarter volatility --- 1.30*** 0.20 Disagreement index, 55 forecasters ++ Full quarter volatility --- 1.12*** 0.15 Disagreement index, 90 forecasters ++ Deadline adjusted volatility --- 1.03*** 0.15 Disagreement index, 71 forecasters ++ Deadline adjusted volatility --- 1.07*** 0.17 Disagreement index, 55 forecasters ++ Deadline adjusted volatility --- 0.91*** 0.13 Disagreement index, 35 forecasters ++ Deadline adjusted volatility --- 0.61* 0.05 Note: ***, **, and * denote statistical significance at 1%, 5%, and 10% levels, respectively; t-tests are based on standard errors corrected using the Newey-West procedure. Table 2: Regression of disagreement index with respect to ex-post volatility, horizons 0,1 and 2. Even if disagreement is imperfectly correlated with volatility, the quality of the regressions presented in Table 2 is good. These regressions show that the larger is the group of forecasters, the greater is the correlation between disagreement and uncertainty, with an R2 increasing from 0.40 to 0.62. When considering the group of 90 forecasters, disagreement and volatility move in opposite directions in 13 quarters. As emphasized by Bowles et al. (2007), even though the overall level of uncertainty may have increased, it is quite possible that forecasters may at the same time 9 increasingly agree with each other about the most likely outcome. This may have occurred between 2008Q2 and 2008Q3: the disagreement index decreased (with forecasters predicting a continuation of the sharp increase observed during the previous quarters) whereas the realized volatility increased (with a fall in the oil price). On the contrary, after 2012 Q2 the disagreement index increased, with exacerbating divergence between forecasters’ assessments of economic outlook or geopolitical issues, while market was becoming less volatile. 5. Calibration of volatility based on disagreement Since every forecast considered is a price averaged over a quarter, the observed disagreement is not directly comparable to oil price volatility. Using a simple benchmark model for the oil price allows us to translate the observed disagreement into volatility. The distribution of forecasts provided at a given date for a given horizon may be interpreted as the distribution of the average price over the corresponding period. Examining the standard deviation of the forecasts is therefore equivalent to examining the standard deviation of the average price. The question is consequently: what is the relationship between the standard deviation of the average price and the volatility of the underlying price? Under the standard assumption that the oil price follows a geometric Brownian motion, we infer the price volatility implied by this distribution. Let us assume that the oil price follows a geometric Brownian motion, with drift and volatility (expressed on a weekly basis). Let h denote the horizon of the forecast, with n being the number of prices observed during the forecasted quarter (typically around 12 for weekly data) and j being the number of prices observed from the start of the current quarter until the end of the quarter preceding the forecasted horizon. Let be the average oil price for horizon h, we have: (9) The forecasters are assumed to know the oil price until the date d. By denoting the price realized at date d as , for we have: (10) (11) (12) Where , respectively. and We first consider that denote the expected value, the variance and the covariance . We have: 10 (13) (14) can be determined from () by considering that is given by the mean forecast for horizon h. We have: (15) gives: (16) With: (17) (18) With simple manipulations, (16) gives: (19) By considering that is given by the variance of forecasts, we can infer the value of . This equation has one solution only, since the right-hand side of (19) is an increasing function of . If we consider that the last price observed by the forecasters is the first of the current quarter, i.e. , then a full-quarter disagreement-based volatility can be inferred from () and (). Alternatively, if we assume that the last price observed by the forecasters is the last price realized before the deadline to send back the filled questionnaire to the ECB, then a deadline-adjusted disagreement-based volatility can be inferred from () and (). 11 Let us now turn to the case where the forecasted quarter is the contemporaneous one. The total number of periods is n, but the price is known until d. We therefore have: (20) (21) The variance of gives: is obtained by replacing in (19) j and n by d and respectively, which (22) 6. Disagreement-based volatility from ECB Surveys Figure 7 : Ex-post volatility and disagreement-based volatility (full quarter, group of 90 forecasters, horizon 0) 12 Figure 8: ex-post volatility and disagreement-based volatility (deadline adjusted, group of 90 forecasters, horizon 0) Figure 9: Ex-post volatility versus disagreement-based volatility, full quarter H0 For every horizon and group of forecasters, we use regressions of the following types: (23) 13 (24) Where and are the full-quarter and deadline-adjusted disagreement-based volatilities respectively. Tables 3 gives the results of the regressions of type (). The R2 are similar to those found when disagreement was regressed against ex post full-quarter volatility. All coefficients, except one, are strongly significant. Exogenous variable Constant Slope R2 Disagreement based volatility, 90 forecasters 0.01** 1.15*** 0.61 Disagreement based volatility, 71 forecasters 0.01** 1.15*** 0.58 Disagreement based volatility, 55 forecasters 0.01 1.18*** 0.51 Disagreement based volatility, 35 forecasters 0.01** 1.07*** 0.48 Disagreement based volatility, 24 forecasters 0.02*** 1.01*** 0.47 Note: ***, **, and * denote statistical significance at 1%, 5%, and 10% levels, respectively; t-tests are based on standard errors corrected using the Newey-West procedure. Table 3: Regressions () of ex-post volatility against disagreement-based volatility (horizon 0, full quarter) Exogenous variable Constant Slope R2 Disagreement based volatility, 90 forecasters 0.01** 0.70*** 0.35 Disagreement based volatility, 71 forecasters 0.02** 0.68*** 0.30 Disagreement based volatility, 55 forecasters 0.02** 0.67*** 0.26 Disagreement based volatility, 35 forecasters 0.02*** 0.62*** 0.27 Disagreement based volatility, 24 forecasters 0.02*** 0.71*** 0.36 Note: ***, **, and * denote statistical significance at 1%, 5%, and 10% levels, respectively; t-tests are based on standard errors corrected using the Newey-West procedure. Table 4: Regressions () of ex-post volatility against disagreement-based volatility (horizon 0, deadline adjusted) The disagreement-based volatility formula captures a non-linear relationship between the disagreement index and the underlying volatility. As a result, for all regressions against deadlineadjusted volatility, the null hypothesis of a normal distribution of the residuals is not rejected by the Jarque-Bera test at a 10% significance level (which was not the case in Section 3). 14 7. Conclusion We find strong statistical evidence that when disagreement or divergence between oil-price forecasters increases, the future oil price is more volatile. This may lead to consider the disagreement index, or the disagreement-based volatility, as an informative index for future volatility in oil prices. In other words, could one of these indices be a good predictor of oil price volatility? In this respect, they can be tested against volatility implied from derivatives markets. An interesting issue would be to determine if the volatility derived from disagreement contains incremental information, relative to the volatility priced by option markets. One might suspect that the myriads of agents interacting on these markets should reveal more about volatility than the level of disagreement between ninety forecasters. To ascertain this is left for future research. References Bomberger, W.A., 1999. Disagreement and uncertainty: A reply to Rich and Butler. Journal of Money, Credit, and Banking, 31, 273-276. Bomberger, W.A., 1996. Disagreement as a measure of uncertainty. Journal of Money, Credit, and Banking 28, 381-392. Dovern, J., Fritsche, U., and Slacalek, J., 2009. Disagreement among forecasters in G7 countries. ECB Working Paper Series 1082. Giordani, P., and Soderlind, P., 2003. Inflation forecast uncertainty. European Economic Review 47, 1037-1059. Kajal, L., and Sheng, X., 2009. Measuring forecast uncertainty by disagreement: The missing link. Journal of Applied Econometrics, forthcoming. Matar W., Al-Fattah, S., Atallah, T. and Pierru, A., 2013. An introduction to oil market volatility analysis. OPEC Energy Review, forthcoming. Patton, A. J., and Allan T., 2008a. The resolution of macroeconomic uncertainty: evidence from survey forecasts. University of Oxford, mimeo. Patton, A., J., and Allan T., 2008b.Why Do Forecasters Disagree? Lessons from the term structure of cross-sectional dispersion. University of Oxford, mimeo. Piersdzioch, C., Rulke J.C., and Stadtmann, G., 2010. New evidence of anti-herding of oil-price forecasters. Journal of Energy Economics 32, 1456-1459. Reitz, S., Rulke, J.C., and Stadtmann, G., 2012. Nonlinear expectations in speculative markets – evidence from the ECB Survey of Professional Forecasters. European University Viadrina Frankfurt, Discussion Paper No. 311. Rich, R.W. and J.S. Butler. 1998. Disagreement as a measure of uncertainty: A comment on Bomberger. Journal of Money, Credit, and Banking 30, 411-419. Sadorsky, P., 2006. Modeling and forecasting petroleum futures volatility. Energy Economics 28, 467–488. Silkos, P., 2012. Sources of disagreement in inflation forecasts : An international empirical investigation. Journal of International Economics, forthcoming. 15 Appendix Number of Forecasts H0 – H3 H4 37 or more 30 or more 20 or more 10 or more At least one 24 35 55 71 90 na 11 45 61 85 Table A1. Number of forecasters providing a minimum number of forecasts, per horizon Figure A1: Number of available forecasts per quarter, horizons H1-H4 16 Figure A1: ECB SPF median forecast and actual average price Endogenous variable Full quarter volatility Deadline adjusted volatility Full quarter volatility Deadline adjusted volatility Full quarter volatility Deadline adjusted volatility Full quarter volatility Deadline adjusted volatility Full quarter volatility Deadline adjusted volatility Full quarter volatility Deadline adjusted Exogenous variable Group of forecasters Horizon Constant Slope R2 Disagreement based full quarter volatility Disagreement based deadline adjusted volatility Disagreement based full quarter volatility Disagreement based deadline adjusted volatility Disagreement based full quarter volatility Disagreement based deadline adjusted volatility Disagreement based full quarter volatility Disagreement based deadline adjusted volatility Disagreement based full quarter volatility Disagreement based deadline adjusted volatility Disagreement based full quarter volatility Disagreement based deadline 90 0 -0.34 0.81*** 0.44 90 0 -1.36*** 0.56*** 0.26 90 1 -0.56 0.67*** 0.38 90 1 -0.93* 0.56*** 0.26 71 0 -1.43*** 0.54*** 0.23 71 0 -0.37 0.80*** 0.42 71 1 -0.63 0.62*** 0.35 71 1 -0.99* 0.55*** 0.24 55 0 -0.57* 0.74*** 0.37 55 0 -1.53*** 0.51*** 0.21 55 1 -0.74* 0.60*** 0.32 55 1 -1.04** 0.53*** 0.23 17 volatility adjusted volatility Full quarter volatility Disagreement based full quarter volatility Disagreement based deadline adjusted volatility Disagreement based deadline adjusted volatility Disagreement based deadline adjusted volatility Disagreement based full quarter volatility Disagreement based deadline adjusted volatility Deadline adjusted volatility Deadline adjusted volatility Deadline adjusted volatility Full quarter volatility Deadline adjusted volatility 35 0 -0.92** 0.63*** 0.32 35 0 -1.75*** 0.43*** 0.19 35 1 -1.20** 0.49*** 0.22 24 0 -1.70** 0.43*** 0.23 24 1 -1.04** 0.51*** 0.3 24 1 -1.25** 0.47*** 0.23 Note: ***, **, and * denote statistical significance at 1%, 5%, and 10% levels, respectively; t-tests are based on standard errors corrected using the Newey-West procedure. Table 1: Regression of log of ex-post volatility against log of disagreement-based volatility 18
© Copyright 2026 Paperzz