Are official GDP growth forecasts for Latin American and Caribbean countries downward biased?

Abstract

This article presents a comparative analysis of GDP growth forecasts for 19 Latin American and Caribbean (LAC) countries, reported by the IMF and ECLAC for the period 2003–2013. Contradicting the general result of the literature that asserts that economic predictions of official organizations tend to be optimistic, our results show that during that period, one year-ahead IMF and ECLAC GDP growth forecasts were downward biased. Furthermore, the analysis of usual goodness of fit measures and accuracy tests show that ECLAC's forecasts performed relatively better than that of the IMF during the period. Another interesting result is that no evidence was found regarding the influence of political type incentives on the bias of GDP forecasts as suggested by some authors. Instead, the results show that downward bias in projections has been strongly influenced by the underestimation of the impact of international economic factors, particularly the drastic increase in commodity export prices, on GDP growth of LAC economies.

Resumen

Este artículo presenta un análisis comparativo de las proyecciones de crecimiento de PBI de 19 economías de América Latina y El Caribe, reportados por el FMI y CEPAL para el período 2003-2013. En contraste con los resultados provenientes de la literatura según los cuales las proyecciones económicas de las organizaciones oficiales tienden a ser optimistas (o sesgadas positivamente), nuestros resultados muestran que las proyecciones de «año a año» del PBI realizadas por el FMI y CEPAL se encuentran sesgadas hacia abajo. Asimismo, a partir de indicadores tradicionales de bondad de ajuste de las predicciones y de test formales de precisión se evidencia un desempeño relativamente superior de los pronósticos elaborados por CEPAL respecto de los reportados por el FMI. Otro resultado interesante es que no se ha encontrado evidencia de la influencia de incentivos de tipo político en el sesgo de las proyecciones de PBI como sugieren algunos autores. En cambio, los resultados muestran que el sesgo hacia la subestimación de las proyecciones durante el período ha estado influenciado por la subestimación del impacto de factores económicos internacionales, en especial, el aumento de los precios internacionales de productos de exportación sobre el PBI de las economías de América Latina y El Caribe.

1

Introduction

During the last years, an extensive debate has emerged regarding the accuracy of economic forecasts reported by a number of international official organizations.¹ In the majority of cases, the debate has focused on the biasedness and efficiency of economic forecasts referred to developed countries. A few of these contributions also include the assessment of forecasts from developing countries but to our knowledge, no study addresses the analysis of economic predictions on specific emerging countries regions. The purpose of this paper is to examine the size and nature of errors in GDP growth forecast reported by the Economic Commission of Latin American and the Caribbean Countries (ECLAC) and the IMF for 19 Latin American and the Caribbean (LAC) countries, during the period 2003–2013. As well, we assess some hypothesis discussed by the recent literature regarding the sources of GDP growth forecast bias, distinguishing those taken from political economy literature from those related with external factors that affect actual behavior of developing countries GDP growth. While the former investigates the relationships between the incentives faced by official organizations, their stakeholder interests and the forecast bias (see for example Aldenhoff (2007) or Dreher and Vaubel (2004), among others) the latter addresses the impact of external shocks on developing country's macroeconomic volatility (see Takagi and Kucur (2006)).

One year-ahead economic growth forecasts of LAC economies produced by ECLAC are reported yearly in the Preliminary Overview of the Economies of Latin American and the Caribbean,² issued by December of each year. On the other hand, IMF forecasts are reported twice a year in the World Economic Outlook (WEO).³ In this paper we contrast the performance of one year-ahead ECLAC's GDP growth forecast with IMF forecasts published in October of each year.

This analysis is relevant for at least three reasons. At first, there could be some common structural factors that may affect “regionally” the performance of forecasts. Secondly, it offers the opportunity to compare the performance of forecasts made by multilateral versus regional official organizations. Thirdly, in some cases, like in LAC, countries can share some institutional similarities that allow the formulation of common political economy hypothesis regarding the origin of forecasts bias.

The paper is organized as follows. In the next section, we examine the traditional measures goodness of fit and we assess the unbiassedness and efficiency hypothesis for GDP growth forecast. In the third section we assess some hypothesis on the factors that may cause forecasts bias in ECLAC and IMF GDP growth forecast errors for LAC countries. Finally, some concluding remarks are presented.

2

Forecast bias, efficiency and accuracy

During the period 2003–2013, LAC countries experienced significant economic growth rates. The main sources of this growth relate with the increase of external demand of raw materials which pushed up the price of commodities, mainly metals, petroleum or agricultural products.⁴Table 1 shows the summary statistics. As mentioned, the sample consisted of 19 LAC countries.⁵ However, some subsamples can be built taking into account geographical proximity of countries and trade union zones. Thus, we included Argentina, Brazil, Paraguay and Uruguay, in Mercosur subsample; Bolivia, Colombia, Ecuador, Peru and Venezuela in Andean Community; Costa Rica, El Salvador, Guatemala, Honduras, Nicaragua, Panama, in Central America, and; Dominican Republic and Haiti in The Caribbean.

Table 1.

Measures of goodness of forecasts.

	MAE		RMSE		U
	IMF	ECLAC	IMF	ECLAC	IMF	ECLAC
All the sample	2.52	2.14	3.36	2.89	0.60	0.52
Mercosur	3.46	2.88	4.29	3.71	0.69	0.60
Andean Community	2.45	2.22	3.42	3.05	0.56	0.50
Central America	1.98	1.55	4.03	3.49	0.51	0.40
The Caribbean	3.38	2.93	4.03	3.49	0.74	0.64

Appendix I shows the summary statistics of actual and projected average growth rates for the overall sample and subsamples. As can be seen, actual average growth rates exceeded IMF and ECLAC forecasts rates, with the only exception of The Caribbean. Indeed, for the overall sample, the differences between actual and forecasted GDP were near a half of a point for ECLAC's and IMF's projections, but the differences were even higher in the case of other subgroups (except The Caribbean). As well, it can be observed that forecasts tended to be relatively smooth compared to actual growth rates. On average actual growth rates volatility were higher than official forecasts’, which is reflected in greater variability coefficients for the overall sample and subsamples.⁶

It is interesting to note that these variability coefficients exceeds largely the values calculated in the same period, for developed economies as United States (1.01), China (0.18), United Kingdom (1.34), Russian Federation (0.99) or France (1.42),⁷ among others. As suggested by the literature (see for example, Takagi and Kucur (2006) and Beach et al. (1999)), these higher rates of volatility (and other factors as the lack of reliable data), makes forecasting of macroeconomic series a difficult task in the developing world. We will return to this issue on next section.

We adopt the usual definition of forecast error, et=GDPFt−GDPRt, where GDPF_t and GDPR_t corresponds to the forecasted and realized GDP growth at year t, respectively. We calculated the traditional measures of goodness of fit for GDP growth forecasts: the Mean Absolute Square (MAE), the Root Mean Square Error (RMSE) and Theil's U.⁸Table 1 shows that these three indicators coincide in reporting a better goodness of fit for ECLAC's forecasts than IMF's, being this conclusion robust for all the regional subsamples.

In order to confirm more formally the intuition provided by Table 1, we test if there is any statistic significant difference between IMF and ECLAC forecasts, through the estimation of a Diebold and Mariano (1995) predictive accuracy test. The null hypothesis tested consisted of H₀: g(e^ECLAC_t) − g(e^IMF_t) = 0; where g () is the loss function associated with the difference between the actual and projected growth.

Table 2 shows the Diebold-Mariano test for the overall sample and subsamples using two alternative loss functions: Mean Square Error (MSE) and MAE. With the only exception of Andean Community and The Caribbean, the results show that null hypothesis of equal accuracy is rejected in favor of ECLAC projections at 5% of significance. This means that ECLAC's forecasts associated loss functions were found significantly lower than IMFs. In other words, with the exception of two subsamples, ECLAC's projections were found relatively more accurate than IMFs, with the exceptions above mentioned. Appendix II shows the plot of both ECLAC and IMF forecast errors for each country.

Table 2.

Mariano-Diebold test on predictive accuracy.

D–M statistic
	Loss function
	MSE	MAE
All the sample	−3.84^***	−3.52^***
Mercosur	−3.24^***	−2.94^***
Andean Community	−1.75^*	−1.79^*
Central America	−2.03^**	−2.28^**
The Caribean	−2.15^**	−1.88^*

*

Significance at 10%.

**

Significance at 5%.

***

Significance at 1%.

Even when the degree of accuracy of IMF and ECLAC forecasts may differ, both projections can suffer of biases in their predictions. Forecast biasedness can be tested and also estimated. Testing biasedness imply evaluating if there exist a significant difference between the actual and forecasted GDP growth series.

In order to assess biasedness, at first, the non parametric Wilcoxon Rank Sum test of equality of medians (Wilcoxon, 1945) was performed, where the null hypothesis is H₀: Median (GDPF_t) − Median (GDPR_t) = 0, against three alternative hypothesis: Ha1: Median (GDPF_t) − Median (GDPR_t) ≠ 0 (two sided); Ha2: Median (GDPF_t) − Median (GDPR_t) > 0 (one sided-right); and, Ha3: Median (GDPF_t) − Median (GDPR_t) < 0 (one sided-left). Table 3 shows that the null hypothesis can be rejected at 5% of significance against the two sided alternative for all the sample and subsamples the alternative hypothesis, for both ECLAC and IMF forecasts, with the only exception of The Caribbean. As well, null hypothesis can also be rejected against Ha3 but not against Ha2, again with the only exception of The Caribbean. Thus, with the only exception of The Caribbean, this non parametric test permit to state that medians of IMF and ECLAC GDP projections are in general downward biased compared with realized GDP at 5% of significance.

Table 3.

Wilcoxon test of equality of medians (projected versus actual GDP) (p values).

	Two tailed test (two sided)		One sided (right)		One sided (left)
	IMF	ECLAC	IMF	ECLAC	IMF	ECLAC
All the sample	0.00	0.00	1.00	1.00	0.00	0.00
Mercosur	0.05	0.05	0.98	0.98	0.05	0.05
Andean Community	0.01	0.00	1.00	1.00	0.01	0.00
Central America	0.02	0.04	0.99	0.98	0.02	0.04
The Caribbean	0.82	0.68	0.32	0.50	0.82	0.68

An usual method for quantifying the amount of the bias, consist of estimating the following regression:

where α is a constant and u_t a stochastic disturbance.

Table 4 shows the results of estimations of the pooled least squares regressions of (1), evidencing at 5% of significance that, on average, IMF and ECLAC's forecasts exhibit a downward bias of around a half a point of GDP growth. In the case of specific subsamples, similar results are observed for ECLAC's estimates of Central America and Andean Community, but with even higher levels of bias. In the case of IMF, the intercept coefficient for these subsamples was not found significant. As well, results for The Caribbean region were not found significant both for ECLAC's and IMF's forecast errors.

Table 4.

Tests for biasedness.

	Pooled regression		Fixed effects regression		Bootstrapping (pooled reg.)
	IMF	ECLAC	IMF	ECLAC	IMF	ECLAC
All the sample	−0.51^**	−0.52^***	−0.51^**	−0.52^***	−0.51^**	−0.52^***
	(0.23)	(0.20)	(0.22)	(0.19)	(0.24)	(0.20)
Mercosur	−1.22^*	−1.00^*	−1.22^**	−1.00^*	−1.22^**	−1.00^*
	(0.63)	(0.55)	(0.60)	(0.54)	(0.60)	(0.54)
Andean Community	−0.81^*	−0.83^**	−0.81^*	−0.83^**	−0.81^*	−0.83^**
	(0.08)	(0.40)	(0.47)	(0.41)	(0.45)	(0.40)
Central America	−0.53^*	−0.54^**	−0.53^*	−0.54^**	−0.53^*	−0.54^**
	(0.10)	(0.24)	(0.30)	(0.22)	(0.30)	(0.24)
The Caribbean	0.57	0.29	0.57	0.29	0.57	0.29
	(0.94)	(0.82)	(0.73)	(0.62)	(0.93)	(0.81)

Note: Standard errors inside parenthesis.

*

Significance at 10%.

**

Significance at 5%.

***

Significance at 1%.

These results are robust to diverse estimation methods. Bootstrapping over 1000 replications of the pooled regressions show similar levels of significance and increases in some cases, as in IMF's forecast for Mercosur. Fixed effects estimations also showed similar results and levels of significance that pooled regression.

Forecast efficiency implies that errors should be unbiased and not be related with information available at time of formulating the predictions. Following Holden and Peel (1990) and Barrionuevo (1996) a weak version of this last condition can be tested estimating the significance of (i) the relation between the forecast error of current and previous period (ρ-test) and (ii) the relation between the forecast error and the forecast of the variable (β-test). Both tests are performed through the estimation of the following equations:

Efficiency condition implies that in (2) and (3) both ρ and β be zero.

Table 5 shows β and ρ estimates obtained from pooled regressions of (2) and (3). As can be seen, for ‘all the sample’ IMF estimates efficiency hypothesis it is rejected. Indeed it can be observed that the hypothesis of dependence of forecasting errors from the forecast itself and from past errors is supported with 5% of significance. As well, in the case of some IMF and ECLAC regional subsamples, efficiency hypothesis is partially rejected. For example, for IMF estimates β-test efficiency hypothesis is rejected in the case of Mercosur and for ECLAC estimates the same occurs in the case of the Andean Community. On the other hand, ρ-tests efficiency hypothesis is rejected for IMF predictions in the case of The Caribbean region while for ECLAC's this hypothesis is rejected in the Central American regression. However, these results are not robust to other estimation methods (see Appendix III). In the case of fixed effects estimation, IMF inefficiency conclusions only holds for the β-test but for the ρ-test. As well, in the case of the bootstrapping estimation for the IMF's forecast errors, β-test significance of results disappears for ‘all the sample’ and only holds for Mercosur.

Table 5.

Tests for efficiency (pooled regression).

	β-Test		ρ-Test
	IMF	ECLAC	IMF	ECLAC
All the sample	0.35^**	−0.13	0.15^**	0.12^*
	(0.14)	(0.11)	(0.07)	(0.07)
Mercosur	0.95^***	0.40	0.10	0.13
	(0.29)	(0.27)	(0.17)	(0.16)
Andean Community	−0.33	−0.78^***	−0.10	−0.15
	(0.31)	(0.24)	(0.13)	(0.13)
Central America	0.14	−0.26^*	0.25^*	0.30^**
	(0.21)	(0.15)	(0.12)	(0.12)
The Caribbean	0.58	0.22	0.53^**	0.38
	(0.43)	(0.36)	(0.21)	(0.24)

Note: Standard errors within parenthesis.

*

Significance at 10%.

**

Significance at 5%.

***

Significance at 1%.

Summarizing, for ‘all the sample’ estimations, unbiasedness null hypothesis is rejected for both organizations but we could not find robust conclusions regarding the lack of weak efficiency. Evidence of bias is also strong in the case of some subsamples for ECLAC's error forecasts, while in the case of IMF no evidence was found. As well, in the case of the selected subsamples, efficiency hypothesis cannot be rejected for both reporting organizations.

This downward bias found for ECLAC and IMF's GDP forecasts contradicts the finding result of some authors (see, for example, Aldenhoff (2007), Beach et al. (1999), Barrionuevo (1993)), that asserts that economic predictions of official organizations tended to be optimistic. In order to explore some plausible explanations regarding these contrasting results, in the following section we evaluate some alternative hypothesis regarding the causes of this bias in the case of LAC countries.

3

On the determinants of forecasting errors

As mentioned before, efficiency means that prediction errors are not related with the information available at the time when forecast is made. In that sense, efficiency tests could be generalized including not only current information regarding the predicted variable or the past forecast errors, as was made in previous section (in specifications (2) and (3)), but also incorporating a broader set of variables.

The performance of LAC economies was quite positive during the period 2003–2013, averaging a 4.4 percent of GDP annual growth. As documented by Camacho and Perez-Quiroz (2013), one the main economic drivers that contributed to explain this result is the relatively high level of international prices of commodities, pushed by the important economic growth of Asian economies like China and India during the same period. Appendix III shows the evolution of six international commodity prices (copper, soy, banana, fishmeal, silver, crude petroleum) exported by LAC countries. In all cases, it can be observed a significant raise of these prices approximately since 2000, achieving the highest levels in last forty years.

As mentioned above, the literature on the determinants of prediction errors has focused either on the statistical properties of forecasting errors or on testing political economy hypothesis referred to the incentives faced by the organizations that report the projections. Political economy hypothesis are based in the notion that forecasts are influenced by economic or political interests of the organization that reports the results and the interests of governments. In the case of IMF, the most common hypothesis assessed is related with the role and influence of member States on the governance structure of these international organizations. The variables most often used to specify the political influence on forecast have been the existence of an outstanding loan agreement and the electoral cycle. In the case of the first factor, the argument behind this hypothesis is that better economic indicators could help to legitimize the decision of lending organizations of approving or maintaining a loan agreement with a specific country. In the case of the electoral cycle, as suggested by Aldenhoff (2007), international organizations like IMF can be interested in getting and keeping the economic and political support from their members. Following the evidence provided by Dreher and Vaubel (2004), which proves that there is a strong positive relationship between electoral cycle and the amount of new net credits of IMF; Aldenhoff (2007) argues that optimistic forecasts can be used by this organization in exchange of ‘weak control’ from their principals.

Finally, in the case of ECLAC's – which is not a financial organization and belongs to a broader organization, the United Nations – we use the variable “country host” in order to assess whether the fact that the place where headquarters of this organization are located can be reflected in an optimistic bias in GDP predictions. The hypothesis in this case is simple: organization's staff can be subject to relatively more pressure from Governments of host countries compared to non-host countries. ECLAC has three subregional headquarters, located in Mexico, Chile and Trinidad and Tobago.⁹

In contrast with political economy hypothesis, less attention has received the discussion of factors related with the nature of the predicted variable. In the case of developing countries, GDP structure is characterized by being strongly dependent on the production of commodities. A number of studies relate these characteristics of the production patterns of developing countries and the degree of volatility of GDP growth. Indeed, UNCTAD (2012), Deaton and Miller (1995) or Easterly et al. (1993) have highlighted the critical influence of external factors, like export prices or terms of interchange, among others, on developing countries economic performance. A central idea that underlies these studies is that the economic dependence on commodity exports makes developing countries especially vulnerable to international volatility in commodity prices. Depending on the magnitude of this dependence, in terms of the share of natural resources on GDP or exports, commodity price volatility could be a source of macroeconomic volatility.¹⁰

Takagi and Kucur (2006) assess the bias of IMF forecast of GDP and inflation for various regions, Latin America included, for the period 1994–2003. One important conclusion they find is that during this period Latin America and Africa's IMF forecasts suffered of an optimistic bias, influenced by two main external economic factors: shocks in the real interest rates of United States and oil prices.

In this study we use three indicators to approximate the impact of these international conditions on GDP: the export unit value, the export purchasing power and the real interest rates. The first one is an index (2010 = 100) calculated by ECLAC for each country from the variations of export prices weighted by a structure of the country's export supply.¹¹ The second indicator, results from dividing total exports between an index of unit value of imports (2010 = 100). Both indexes are computed in variations. Thirdly, in order to assess Takagi and Kucur's findings for previous periods, we include also as explanatory variables the real interest rate in the United States.

In order to test the empirical relevance of the hypothesis related to the political factors that could influence on ECLAC and IMF GDP forecasts versus the influence of external factors on GDP of LAC countries, we estimate the following equation:

where PEtj and EFtj are vectors of political economy and external factors, respectively, which influence forecast errors on LAC countries’ economic growth predicted by organization j, where j = ECLAC, IMF. PEtj vectors are defined as follows: PEtECLAC=(Country Host, Change Government) and PEtIMF=(Outstanding Loans, Change Government), while EFtj=(ΔExport Unit Value, ΔExports Buying Power, US Real Interest Rates).

The combination of political economy and external factors variables results in the estimation of four equations for each organization's predictions, showed in Table 6.

Table 6.

Determinants of GDP growth forecasts.

			PEt		EFt
Specification	GDPFtECLAC	et−1	Country host	Chang. gov.	d (export unit value)	d (export buying power)	US interest rates
i.	−0.13	0.07	1.11^*		−6.84^***		0.19
	0.13	0.07	0.64		1.92		0.16
ii.	−0.20	0.10	0.89			−0.02^***	0.13
	0.13	0.08	0.65			0.01	0.16
iii	−0.13	0.08		0.35	−0.07^***		0.19
	0.13	0.07		0.48	0.02		0.22
iv.	−0.19	0.11		0.43		−0.02^***	0.14
	0.13	0.07		0.49		0.01	0.16

			PEt		EFt
Specification	GDPFtIMF	et−1	Outstanding loans	Chang. gov.	d (export unit value)	d (export buying power)	US interest rates
v.	0.28^*	0.14^**	0.02		−0.10^***		0.20
	0.15	0.07	0.01		0.02		0.18
vi	0.30^**	0.20^***	0.02			−0.03^***	0.09
	0.15	0.07	0.01			0.00	0.18
vii.	0.26^*	0.14^**		0.53	−0.10^***		0.19
	0.15	0.07		0.54	0.02		0.18
viii	0.28^*	0.20^***		0.59		−0.03^***	0.09
	0.15	0.07		0.56		0.01	0.18

Note: Standard errors inside parenthesis.

*

Significance at 10%.

**

Significance at 5%.

***

Significance at 1%.

As can be seen, in both ECLAC's and IMF's forecast errors, real external factors were found significant, evidencing that increases in the value of exports unit value or export buying power, contribute to explain the downward bias of GDP predictions. For example, in the case of IMF's predictions, results show that an increase in 1% in the variation of the export unit value index contribute with a of 0.1 points downward bias in GDP growth forecasts. As well, an increase in 1% of the Export Buying Power Index explains a decrease in 0.03 points of GDP growth forecasts. In contrast, no evidence was found regarding the impact of United States real interest rates¹² on ECLAC's and IMF's forecast errors.

In the case of political economy variables, neither IMF's nor ECLAC's proxies showed a significant influence on GDP growth forecast errors. In the case of IMF, these results holds despite the fact that those countries with the better performance during the period like Panama, Argentina, Dominican Republic or Peru, held outstanding GRA¹³ or IMF Loans.¹⁴ In the case of ECLAC political economy proxy, host countries do not show a particular bias with respect to other members. Finally, no evidence has been found between the forecasts bias and the political cycle¹⁵ both in the case of ECLAC and IMF.

These results contrast with Takagi and Kucur's earlier findings in at least two directions. Firstly, the bias found for Latin American countries was pessimistic – not optimistic- both for ECLAC's and IMF's forecasts. Secondly, the determinants of this bias were not related with international monetary factors like US interest rates and political incentives of official organizations; but with real external factors associated with unanticipated increases of the world demand of raw materials and their impact on the prices of exports and terms of interchange of LAC countries.

Concluding remarks

During the period 2003–2013, LAC economies experienced an unprecedented growth boosted by a boom of international commodity prices, led by an important increase of exports of natural resources. The evidence shown in previous section indicates that forecasts reported by ECLAC and IMF failed partially to anticipate the importance and magnitude of the impact of these external factors on GDP of LAC economies. An analog but opposite phenomena has been observed in the case of GDP forecasts in times of recessions reported by the official and private organizations. Examples of that are found in Barrionuevo (1993) in the case of IMF projections¹⁶ or Loungani (2000) in the case of Consensus Forecast.¹⁷

However, some important future research avenues can be identified. At first, even taking into account the difficulties in predicting economic recessions or booms, it could be important to inquire whether the dependence of LAC countries on natural resources exports and commodity prices, makes their macroeconomic variables more volatile and difficult to predict. Secondly, it could be useful to confirm if, as in the case of this paper, the performance of the predictions made by regional official organizations are in general better than those made by multilateral organizations. Thirdly, it could be interesting to inquire if political cycle hypothesis as determinants of official forecasts, can be more likely and relevant in the case of developed countries which can have a more important and decisive role as principals of reporting official organizations, compared with developing countries.