DESIGNING AN EARLY WARNING SYSTEM FOR DEBT CRISES
BY ALESSIO CIARLONE * AND GIORGIO TREBESCHI *
Abstract
This paper develops an early warning system for debt crises broadly defined as episodes of
outright default or failure of a country to be current on external obligations. A multinomial
model is applied, that allows to differentiate between three regimes: a ‘tranquil’ period, a
‘crisis’ state and an adjustment or ‘post-crisis’ phase. The model, estimated using a large set
of macroeconomic variables, is able to predict 76 per cent of entries into crisis while sending
36 per cent of false alarms and has rather good out-of-sample performance. Finally the paper
tries to integrate the analysis based on macroeconomic variables with the approach based on
risky market instruments.
JEL Classification numbers: H63, E66.
Keywords: emerging markets, early warning system, debt crises, default.
*
Bank of Italy, International Relations, via Milano 64, 00179 Rome. Telephone: 0039-06- 47925321/5584; fax:
0039-06-47925324; email addresses: [email protected], [email protected].
1. INTRODUCTION (•)
Emerging market economies have experienced crises, driven by their poorly developed
financial systems, volatility of macroeconomic policies, weak banking sector, high
dependence on external capital flows, uncertain growth prospects; such crises had disruptive
effects on these economies.
For this reason, both the academic and the official sector have felt the importance of
developing models that can not only identify weaknesses and vulnerabilities in emerging
market economies, but also send timely and correct signals about the onset of a financial
crisis, the so-called early warning systems (EWS).
Most of the EWS models developed so far have tried to signal the onset of currency and
banking crises, both individually or jointly determined (so-called twin-crises). The seminal
papers in the field are those written by Kaminsky, Lizondo and Reinhart (1998) and by
Frankel and Rose (1996), whereas a thorough review of such category of models can be found
in Berg, Borensztein and Pattillo (2004).
Until now, however, little work has been done on ‘debt’ crises, which can be broadly
defined as episodes of default or failure to be current on external obligations. The time span
from 1994 onwards has been characterized by large sovereign and corporate defaults, or debt
servicing difficulties on foreign currency-denominated bonds.
Currency and debt crises might be generated by common factors, such as unfavorable
macroeconomic developments, a deterioration in external financing conditions (e.g. a sudden
reduction in capital flows or a sharp upsurge in their cost) or a increase in the extent of
international investors’ risk aversion.
Notwithstanding the previous considerations, currency and debt crises do remain quite
distinct events, in fact:
1)
the two types of crises are not perfectly correlated, as shown in Sy Amadou (2003), in
the sense that it is possible to have a currency crisis which is not associated with a debt
crisis, as it is conceivable that a country may fall into arrears or default on its external
debt without any major disruption in the exchange rate, as happened in Pakistan in
1999;
2)
it is not clear what the causal relationship should be: in fact, one could expect a sharp
depreciation of the exchange rate as a response to an excessively high growth of the
external debt or a rapidly worsening scenario for the country’s financing needs; under
•
We would like to thank our collegues Mr. Antonello Fanna, for his intellectual and moral support, and Ms.
Paola Paiano, for her valuable research assistance. The paper benefited from the useful comments of an
anonimous referee. The opinions expressed hereinafter do not reflect those of the Bank of Italy. Any errors and
omissions remain our own responsibility.
such a scenario, investors might not trust the government’s ability to face its external
obligations and therefore start selling off assets denominated in that particular currency.
The literature on the empirical determinants of a debt crisis is quite small compared
with the large body of theoretical and empirical work on currency and banking crises. A broad
classification is between models which are based on the evolution of a particular set of
macroeconomic variables leading to the build-up of a crisis, and models that extract
information from financial data and market prices for widely traded financial instruments
such as sovereign bonds or, more recently, credit default swaps (CDSs). As recognized by the
IMF itself (2002), any macro-based model should be complemented with information on
market expectations extracted from bond spreads as well as from CDSs. A detailed
description of the afore mentioned categories of EWS for debt crises can be found in Ciarlone
and Trebeschi (2004).
The rest of this paper is organized as follows: section 2 reports the definition of debt
crisis, the data set and the event study analysis; section 3 presents the two econometric
specifications, the binomial and the multinomial logit, both based on macro variables; section
4 is devoted to the analysis of CDSs spreads with the aim to extract early warning signals.
Section 5 concludes.
2. DEBT CRISIS, MACROECONOMIC VARIABLES AND EVENT STUDY ANALYSIS
The main concern of a researcher is to define what a debt crisis really is: the actual
problem is that such event may take on different forms, ranging from an outright default
declared by an insolvent country on part or all of the stock of external or public debt, to debtservicing difficulties determined more by illiquidity than by insolvency. These increasing debt
servicing difficulties might well be signaled by the accumulation of interest or principal
arrears (e.g. Detragiache and Spilimbergo, 2001) or by a worsening of the market evaluation
of a country’s creditworthiness, as measured by an increasing spread over US or euro
denominated bonds (e.g. Pescatori and Sy, 2003). The very recent past has also witnessed
many events in which outright default has been prevented by large aid packages provided by
the IFIs (e.g. Turkey in 2001 and Brazil in 2002) or by restructuring agreements with the
private sector (e.g. Uruguay in 2002).
In what follows, we will describe the procedure exploited to define the dependent
variable, i.e. the occurrence of a debt crisis, the macroeconomic variables that we think may
be significant in determining such event and their behaviour before and after the occurrence
of a crisis. All of this will serve as a necessary background for the econometric analysis,
which is introduced in section 3.
-3-
A new definition of debt crisis
Our debt crisis indicator is derived from data provided by the World Bank’s Global
Development Finance database (GDF), the IMF’s International Financial Statistics database
(IFS), the Paris Club website and internal sources.
More precisely, we have defined a debt crisis as the event when one or more of the
following conditions occur:
1) a country has officially declared a moratorium on public or external debt payments;
2) a country has incurred a missed payment of interest and/or principal on external
obligations towards official and commercial creditors which adds up to more than 5
per cent of the debt service ratio paid by year-end;
3) a country has accumulated arrears of interest and/or principal on external obligations
towards official and commercial creditors, which add up to more than 5 per cent of
the total external debt outstanding by year-end;
4) a country has signed a debt restructuring or rescheduling agreement with official
and/or commercial creditor;
5) a country has received a large assistance package from the IMF, where large is
defined as access in excess of 100 per cent of quota.
Table 1 hosts the chronology of crises from 1980 to 2002 for 28 emerging market
economies with significant market access: (1) for each country in the sample, the table shows
the number of crisis episodes experienced as well as their average length. For the sample as a
whole, we have found out 44 debt crises with an average length of almost 7 years.
1
In order to construct our sample, we first started with the set of 31 emerging market economies included in the
JPMorgan EMBI Global Diversified. As a second step, four countries (Bulgaria, Ivory Coast, Lebanon and
Ukraine), which lacked comprehensive macroeconomic time series, were taken out whilst Korea was included.
-4-
Table 1 Countries and debt crisis episodes, 1980 - 2002
Country
Number of crises
Years in crisis
Average length
Crisis episodes
Argentina
2
15
7.5
1983 – 1995; 2001 - …
Brazil
2
16
8.0
1983 – 1993; 1998 - …
Chile
1
8
8.0
1983 – 1990
China
---
---
---
---
Colombia
1
1
1.0
1988
Dominican Rep.
2
19
9.5
1982 – 1999; 2002 - …
Ecuador
1
18
18.0
1983 – 2000
Egypt
2
13
6.5
1980 – 1991; 1995
El Salvador
2
5
2.5
1984; 1989 – 1992
Hungary
---
---
---
---
Indonesia
1
6
6.0
1997 - …
Korea
3
6
2.0
1980 – 1981; 1984; 1997 – 1999
Malaysia
---
---
---
---
Mexico
2
13
6.5
1982 – 1992; 1995 – 1996
Morocco
2
11
5.5
1983 – 1992; 1999
Nigeria
1
17
17.0
1986 - …
Pakistan
2
6
3.0
1981 – 1982; 1998 – 2001
Panama
1
13
13.0
1983 – 1995
Peru
3
16
5.3
1980; 1983 – 1996; 2000
Philippines
2
9
4.5
1984 – 1991; 1994
Poland
1
13
13.0
1981 – 1993
Russia
1
14
14.0
1989 - …
South Africa
2
6
3.0
1985 – 1989; 1993
Thailand
2
4
2.0
1981; 1997 – 1999
Tunisia
1
1
1.0
1991
Turkey
2
6
3.0
1980 – 1982; 2000 - …
Uruguay
3
5
1.7
1983; 1986 – 1988; 2002 - …
Venezuela
2
12
6.0
1984 – 1994; 1998
Total
44
6.7
Source: author’s calculations based on data from IMF and World Bank.
Variables used in the econometric specification
The set of independent variables comprises 28 macroeconomic indicators, largely drawn
from the literature on debt sustainability: in particular, these variables essentially measure the
burden of external indebtedness, the amount of resources destined to its service, the foreigncurrency generating capabilities of a country, the external monetary and financial conditions,
the net capital flows.
-5-
Event study analysis
The event study analysis is a useful tool to investigate how explanatory variables
behave around default episodes and to give a graphical interpretation to it. (2) The results of
the event study analysis are shown in Chart 1: the bold horizontal line represents the average
value of the variable during tranquil period, whereas the solid line shows the average value of
the variable during stress time with a 95 per cent confidence interval around it, identified by
the two dotted lines. Time goes from t-3 to t+3, where t is the year in which our debt crisis
indicators is equal to one for the first time, thus signalling a crisis entry.
The event study analysis suggests the following results:
2
-
measures of external debt service are significantly different during crisis periods
with respect to tranquil times: interest payments on external debt, that are on
average 54 per cent of the level of international reserves during tranquil periods,
more than double in the year preceding a crisis and become increasingly larger in
the crisis year and in the following one; total debt service, that includes principal
payments as well, displays a similar pattern, regardless whether it is expressed as a
percentage of international reserves or of exports; total external debt to GDP
increases during the run-up to a crisis and, at t+1, it becomes significantly higher
then its average in tranquil periods;
-
measures of the level of international reserves change significantly during crisis
periods as opposed to tranquil times: in particular, the level of reserves as a
percentage of total external debt drops significantly from 25.2 per cent during
tranquil time to 14.4 in the year preceding a crisis;
-
the degree of openness of the economy, measured as the sum of export and imports
divided by GDP, significantly decreases in the year preceding crisis, to 36.3 per cent
from an average of 47.5 per cent during tranquil periods: this result may be ascribed
to the reduction both in imports growth (from an average of 9.5 per cent in tranquil
period to –1.6 in the year preceding the crisis) and in exports growth (form 8.5 to
0.4 per cent);
-
some measures of short-term debt display interesting features: short-term debt over
international reserves, for example, increases sharply in the run-up to a crisis from
about 220 per cent to 383 per cent.
In order to do that, we took into account crisis-entry years, the preceding and the following three years
identifying, in this way, seven additive dummies. We then run a simple regression of the variable on a constant
and the set of the seven dummies previously determined. The coefficient on the constant represent the average
of the variable in tranquil periods, whereas the coefficients on the dummies give information on the behaviour
of the variable around crisis periods.
-6-
Chart 1: Event study analysis
-7-
-8-
-9-
3. ECONOMETRIC SPECIFICATIONS
In order to find out which variables are significant in determining the onset of a debt
crisis, we have used a logit approach.
More precisely, we have performed two different logit specifications. In the first one,
suggested by the work of Manasse et alia (2003), we allow the independent variables to have
a different impact on the probabilities of ‘entering’ and ‘being’ into a crisis event. In the
second specification, suggested by the work of Bussiere and Fratzscher (2002), we have
constructed a multinomial logit with three, instead of two, regimes: a tranquil period, a crisis
event and an adjustment phase during which the level of the macroeconomic variables comes
back to a more sustainable path. The main difference with Bussiere and Fratzcher’s paper,
beyond that related to the frequency of observations (monthly vs. annual), is that their models
applies to currency, instead of debt, crises.
- 10 -
A two-regimes logit EWS for debt crises
As said before, in this first specification we allow the independent variables to have a
different marginal effect on the probability of ‘entering’ and of ‘being’ into a debt crisis: (3)
more formally, entering into a crisis refers to the first year of the episode, while being into a
crisis refers to the remaining years until the end of the negative event.
The dataset used in the logit regression includes the 28 macroeconomic variables that
have been the subject of the event study analysis: the entire sample runs from 1980 to 2002.
This means that we used macro data from 1980 to 2001 in order to explain the occurrence of
debt crisis events from 1981 to 2002: therefore, the final regression sample contains
something as 616 observations. (4)
Our estimation approach foresees three different steps:
3
4
5
-
first of all, we run logit regressions for each of the 28 variables independently from
one another and we exclude all the variables that turn out to be insignificant in
determining the ‘entering’ or the ‘being’ into a debt crisis, as well as the variables
that, although being significant, have a counterintuitive sign;
-
in a second phase, we run group-wise regressions, i.e. we group in families,
essentially according to their nature, all the variables that got through the first step
and then we run new logit regressions for each of these groups: as in the first phase,
we retained only those variables that turned out to be significant and had the correct
sign;
-
in the final step, we put together all the variables that got through the first and the
second step into a general logit regression. We added back variables that, for
example, have been found to be significant in the literature or that displayed a
particular behaviour in the event study analysis, but were dropped in either the first or
the second step: (5) using a ‘general to specific’ approach in order to reach a
parsimonious model, we again drop all the insignificant variables.
In all the logit regressions belonging to this particular specification each independent variable is entered twice
and is multiplied, respectively, by the factor (crisis-1) or (1-(crisis-1)): if the crisis indicator signaled a ‘one’,
the second factor disappears; the opposite holds if the debt crisis indicator signaled a ‘zero’. The estimated
coefficient on the first factor measures, therefore, the marginal effect of the variable on the probability of being
in crisis in t given that the country was already in crisis in t-1, while the estimated coefficient on the second
factor measures the marginal effect of the variable on the probability of entering into a crisis in t given that
there was no crisis in t-1.
In this way the number of crisis entries decreases from 44 to 40, after dropping 1980 crises for Turkey, Peru,
Korea and Egypt. It must be noticed that all the ‘zeros’ positioned between two ‘ones’ have been turned into
‘ones’, that is in this case a non crisis observation has been considered as a continuation of an ongoing crisis.
This is essentially done in order to try to avoid the so-called omitted-variable bias, which may render
coefficient estimates biased and inefficient leading to the exclusion, in the first stage, of variables that should
be otherwise retained (e.g. Visco, 1978). Another problem that may arise is that variables that are not part of
the true model may be retained because the bias induced by some omitted variable make them look
significantly different from zero. This problem should be mitigated by the fact that the model is estimated in
the final step using a large set of regressors which hopefully includes most of the true variables.
- 11 -
At the end of this procedure, we retain just six variables: interest payments on external
debt scaled to international reserves; the degree of openness to international trade; the export
growth rate; the ratios of total external debt and short term debt to GDP; the ratio of
international reserves to total external debt. Table 2 reports the estimation coefficients.
Table 2 Post group-wise regressions: coefficient estimates
Coefficient
Std. Error
z-Statistics
P Value
-0.413744
0.008795
0.004956
-0.053309
-0.033290
-0.007041
-0.035932
-0.013798
0.585278
0.004020
0.002351
0.023013
0.014782
0.011008
0.014962
0.009327
-0.706920
2.187990
2.108225
-2.316499
-2.252076
-0.672990
-2.401462
-1.479371
0.4796
0.0287
0.0350
0.0205
0.0243
0.5010
0.0163
0.1390
-0.029078
0.033219
0.011477
0.010312
-2.533554
3.221344
0.0113
0.0013
CRISIS(-1)*STDGDP
0.000172
0.052645
0.014029
0.023654
0.012239
2.225622
0.9902
0.0260
(1-CRISIS(-1))* STDGDP
0.013674
0.023555
0.580509
0.5616
Pseudo R-squared
0.543558
Variable
C
CRISIS(-1)*INTDEBTRES
(1-CRISIS(-1))*INTDEBTRES
CRISIS(-1)*RESTED
(1-CRISIS(-1))* RESTED
CRISIS(-1)*OPEN
(1-CRISIS(-1))* OPEN
CRISIS(-1)*EXPGROWTH
(1-CRISIS(-1))* EXPGROWTH
CRISIS(-1)*TEDGDP
(1-CRISIS(-1))* TEDGDP
Testing whether the marginal effects on the probability of entering into a crisis or being
into a crisis are the same led us to reach the final specification, which is reported in Table 3.
Table 3 Logit Early Warning System: final specification
Coefficient
Std. Error
z-Statistics
P Value
-0.589821
0.005756
-0.042032
0.469613
0.002116
0.015103
-1.255973
2.720616
-2.783060
0.2091
0.0065
0.0054
CRISIS(-1)*STDGDP
-0.023255
-0.20966
0.045753
0.065663
0.008513
0.007526
0.007594
0.026415
-2.731524
-2.785927
6.024838
2.485791
0.0063
0.0053
0.0000
0.0129
Pseudo R-squared
0.534649
Variable
C
INTDEBTRES
RESTED
OPEN
EXPGROWTH
CRISIS(-1)*TEDGDP
The probability of entering, or being, into a debt crisis seems to depend essentially on
the behaviour of variables that measure the burden of external indebtedness and the foreigncurrency generating capabilities of a country:
- 12 -
-
the amount of interests a country has to pay on its external debt obligations, scaled
to international reserves, seems to have the same positive marginal effect on the
probability of entering and being into a crisis;
-
measures of total external debt and short term debt, both scaled to the country’s
GDP, seem to have a positive effect on the probability of remaining, rather than
entering, into a debt crisis: high levels of total external debt make exiting from a
crisis more difficult;
-
the amount of international reserves, scaled to the total external indebtedness, has a
negative effect on the probability of entering, and being, into a debt crisis: countries
with high levels of international reserves, therefore, seem to be more sheltered than
others from such negative events;
-
export growth has the same beneficial negative effect on the probability of incurring
a debt crisis: exports, in fact, are an important source of foreign currency, with
which a country could more easily face its foreign-currency denominated debt
obligations;
-
the degree of a country’s openness to international trade can reduce the probability
of entering and being into a debt crisis: countries with strong commercial links with
the rest of the world seem to be less prone to experience a debt crisis, especially if
the occurrence of debt servicing problems are associated with a depreciation of the
exchange rate, which boost exports and squeezes imports.
A first important test to perform in order to evaluate the goodness of fit of the estimation
results is relative to the model’s in-sample predictive ability: a particular cut-off level, or
threshold, has to be identified, above which the predicted crisis probability could be
considered as sending a signal that a negative event is about to occur. The question, now, is
rather how this “optimal” threshold should be calculated: the lower the threshold, the more
signals the model will send, with the danger of sending many false alarms relative to crises
that would never occur; the higher the threshold, the lower the numbers of signals, with the
risk of not being able to capture the onset of a crisis.
Since a precise rule to determine the “optimal” cut-off level does not exist, this problem
is often resolved by classifying an observation as predicting a crisis when the estimated
probability exceeds the in-sample frequency of a crisis. In our case, it would not be correct to
use an ‘overall’ cut-off level for the estimated probabilities: what we are trying to predict, in
fact, is the entry into a crisis event while, by using an overall threshold level, we are implicitly
taking into account the entire set of the predicted crisis events (i.e. all the ‘ones’), without
discriminating among true entries into a crisis and events that are, instead, just the prosecution
of an ongoing crisis.
- 13 -
In the light of this consideration, we use two different thresholds levels for the estimated
probabilities, one to signal the year in which a crisis starts and the other to signal the year
belonging to an ongoing crisis. (6) By the means of this strategy, we are able to correctly
predict 72 per cent of crisis entries (i.e. 29 out of 40); nonetheless, our model now sends
almost 36 per cent of false alarms (i.e. signals that are not followed by an actual crisis
episode).
Table 4 Logit EWS: goodness of fit with two thresholds for the estimated probabilities
Crisis
No crisis
Total
Signal
175
132
307
N. of observations correctly called
67%
No signal
74
235
309
N. of crisis entries correctly called
72%
Total
249
367
616
N. of false alarms
36%
True state
Crisis entries correctly called (29):
Argentina 1983, 2001
Brazil 1983, 1998
Chile 1983
Colombia 1988
Dominican Rep. 1982
Ecuador 1983
El Salvador 1984, 1989
Korea 1984
Mexico 1982, 1995
Morocco 1983
Nigeria 1986
Crisis entries not correctly called (11)
Pakistan 1981, 1998
Panama 1983
Peru 1983
Philippines 1984
Poland 1981
Russia 1989
South Africa 1985, 1993
Turkey 2000
Uruguay 1983, 1896,2002
Venezuela 1984
Dominican Rep. 2002
Egypt 1995
Indonesia 1997
Korea 1997
Morocco 1999
Peru 2000
Philippines 1994
Thailand 1981, 1997
Tunisia 1991
Venezuela 1998
As Table 4 shows, the model is not able to pick up the 1997 crises in Korea and
Thailand, while the signal for Indonesia is just borderline. Those are crises that originated
from the breakdown of fixed exchange rate regimes, compounded by the presence of currency
and maturity mismatches in the banks’ balance sheets and overinvestment in the corporate
sector. The deterioration of the fundamentals in the years before those crises, observable in
the set of macroeconomic variables identified in our paper, was not such as to warrant the
severe turbulence that occurred afterwards: these examples, perhaps, highlight a drawback of
our approach to EWS modeling, i.e. that it does not capture to the full extent this particular
type of crisis transmission mechanism.
The case of Russia deserves some further explanation. Russia experienced a prolonged
period of crisis, according to the definition given in section 2, that started in 1989 and
continued for the following years. This came as a result of a combination of missed payments
and arrears on external obligations in the first years and, at a later stage, of debt restructuring
6
The threshold used for signaling entry into crisis is 11 per cent, while the threshold used for signaling being
into a crisis is 84 per cent: these levels have been calculated, as before, as the in-sample frequencies of crisis
entries (a zero followed by a one) and crisis ongoing (a one followed by a one).
- 14 -
on official debt. In 1998 Russia received a large assistance package from the IMF as well,
during a crisis in which it announced a compulsory restructuring of domestic debt and a 90day moratorium on foreign commercial debt. (7) The logit model correctly picks up the onset
of the crisis in 1989 and gives correct and consistent signals throughout the following years,
including on the 1998 crisis.
The goodness of fit of the logit results has to be probed also, and especially, from the
point of view of its out-of-sample predictive ability: therefore, we have re-estimated the
model for the years from 1980 to 1999 and we have predicted the crisis events from 2000
onwards. As Table 5 shows, while our debt crisis indicator signals the occurrence of five
crisis episodes during this time span (Peru and Turkey in 2000, Argentina in 2001, Dominican
Republic and Uruguay in 2002), our model is able to predict just two of them (Argentina and
Uruguay), which represents a predictive ability of 40 per cent.
Table 5 Logit EWS: out of sample goodness of fit *
Crisis
No crisis
Total
Signal
15
11
26
N. of observations correctly called
77%
No signal
8
50
58
N. of crisis entries correctly called
40%
Total
23
61
84
N. of false alarms
18%
True state
* The model is estimated with data up to 1999 to try to predict entries from 2000 onwards
A multinomial logit EWS for debt crises
In order to improve upon the ability to predict crisis entry, we tried to estimate a
multinomial logit model. This approach has a great advantage, with respect to the simple
binomial logit, in that it allows to construct more than two regimes and, in this way, to model
explicitly the ‘crisis entry’ as opposed to the ‘adjustment’ regime. This approach has been
initially proposed by Bussiere and Fratzscher (2002) to predict currency crises, and has
demonstrated to lead to better econometric results since it is able to deal with the so called
‘post-crisis’ bias. The post crisis bias can be traced back to the different behaviour shown by
macroeconomic variables in the run-up to a crisis as opposed to the period right after a crisis
has erupted, when the adjustment process is taking place: disregarding this fact might cause
problems when estimating the model and lead to biased coefficient estimates.
Moving from the dependent variable as identified in the binomial logit model, we defined
three different regimes according to the rule contained in the following Table 6:
7
Table 1 in the previous section identifies crisis episodes and their length.
- 15 -
Table 6 Regime definition in the multinomial logit model
Original Crisis Definition
(dependent variable in the binomial logit model)
At time t-1
at time t
at time t+1
1
0
0
1
1
0
1
0
0
0
0
1
1
1
0
1
0
0
1
0
1
0
1
1
Regime in the multinomial
Model at time t
Tranquil (Y = 0)
Crisis (Y = 1)
Post – crisis (Y = 2)
The ‘crisis’ period is identified by a situation in which an economy has not been in a
crisis state over the last two years but will be facing troubles in the year ahead; a ‘tranquil’
state is defined as the case in which the economy has never experienced a crisis or has just
exited one at time t-1. All the other cases are treated as ‘adjustments’ or post-crisis periods. (8)
As a first step, the multinomial model is estimated with maximum likelihood, with the
tranquil period as the benchmark, and using as regressors the same set of variables that were
found to be significant when estimating the binomial logit model (see Table 3). As a second
step, the model is re-estimated adding other variables that were, for example, found to be
significant in the literature or that displayed a particularly behaviour in the event study
analysis: this is done, again, as a means to try to avoid the omitted-variable bias.
Table 7 summarises the results obtained from the estimation of the multinomial model. (9)
The first part of the table is the most important as it shows estimates of the coefficients
relative to the likelihood of the economy to entry a crisis within one year as opposed to the
likelihood of remaining in a tranquil state. The second part of the table refers to estimates for
the coefficients that give information about the likelihood that a country continue to be in a
recovery state as opposed to the likelihood of returning to a tranquil state.
8
9
As Table 6 shows, in order to identify the multinomial logit regime three observations of the dependent
variable from the binomial specification are needed. This automatically reduces the number of observation
available for estimation. In particular, the entries into default go down from 40 in the binomial logit model to
37 in the multinomial, after dropping the crisis entries in 1981 for Pakistan, Poland and Thailand. The lack of
data for 1979, in fact, does not allow us to unambiguously determine whether 1980 is a crisis as opposed to a
post crisis year.
The reader interested in the technicalities of the multinomial approach are referred to Bussiere and Fratzscher
(2002) for an easy explanation.
- 16 -
Table 7 Multinomial Logit estimation
Variable
Coefficient
Std. Error
Z-Statistics
P-value
-2.264348
0.024091
0.003111
-0.028213
0.628980
0.010661
0.001061
0.011953
-3.600033
2.259816
2.932272
-2.360352
0.0003
0.0238
0.0034
0.0183
-0.616721
-0.059251
0.052872
-0.034504
0.378300
0.010628
0.006529
0.006136
-1.630243
-5.574893
8.097865
-5.622858
0.1031
0.0000
0.0000
0.0000
Pre – crisis period (Y=1)
CONSTANT
TEDGDP
INTDEBTRES
OPEN
Post – crisis period (Y=2)
CONSTANT
RESTED
TEDGDP
OPEN
Pseudo R-squared
0.24
Looking at the results in Table 7, one can see that measures of the level of external debt,
international reserves, debt service, along with the degree of openness of an economy are
significant in determining whether an economy is likely to face troubles.
The first half of Table 7 shows that interest rate payments, expressed as a percentage of
the international reserves, are significant in determining crisis entry and enter with the
expected positive sign: an increase in interest rate payments increase the likelihood of
entering a crisis. The degree of openness of the economy is another variable that appeared
significant in the binomial logit estimation and that is retained in the multinomial logit
specification. The sign of the coefficient is negative as expected, confirming that the more
open to international trade an economy is, the less it is likely to experience a crisis. Finally,
the level of total external debt on GDP appears to be significant at the 5 per cent level and
displays the expected sign, meaning that the higher the level of external debt the more likely
the country is to enter a crisis rather than continue to stay in a tranquil state.
The total external debt on GDP and the degree of openness appear also in the second half
of Table 7, suggesting that those variables are significant in determining not only the crisis
entry but also its length. Table 7 indicates that the level of international reserves, expressed as
a percentage of the total external debt, is significant in determining the likelihood of an
economy to return to a tranquil state, after experiencing a crisis. The coefficient on the
variable is of the right sign, suggesting that during the process of adjustment reserves are
expected to increase while external debt is expected to decrease.
Table 8 shows some data on the goodness of fit. Analogously to the binomial case, the
signals are obtained using two distinct thresholds, one for entering the crisis and another one
for staying in crisis.
- 17 -
Table 8 Multinomial Logit Model: goodness of fit
True state
Correctly
called
Not correctly
signalled
Total
Pre – crisis
28
9
37
N. of observations correctly called
71%
Post – crisis
190
50
240
N. of crisis entries correctly called
76%
Tranquil
197
111
308
N. of false alarms
36%
Total
415
170
585
Crisis entries correctly called (28):
Crisis entries not called (9)
Argentina 1983, 2001
Brazil 1983, 1998
Chile 1983
Colombia 1988
Ecuador 1983
Egypt 1995
El Salvador 1989
Indonesia 1997
Korea 1984
Mexico 1982, 1995
Morocco 1983, 1999
Nigeria 1986
Dominican Rep. 1982, 2002
El Salvador 1984
Korea 1997
Russia 1989
South Africa 1993
Thailand 1997
Tunisia 1991
Venezuela 1998
Pakistan 1998
Panama 1983
Peru 1983, 2000
Philippines 1984, 1994
South Africa 1985
Turkey 2000
Uruguay 1983, 1896, 2002
Venezuela 1984
The in-sample predictive ability of the model shows that the multinomial setting might
represent an improvement upon the results of the simple binomial model. Comparing Table 8
with Table 4, it is evident that the goodness of fit does improve from the binomial to the
multinomial, although not dramatically. The number of observations correctly called increases
from 67 to 71 per cent, the number of crises entries correctly called goes up from 72.5 to 76
per cent. (10) As opposed to the binomial model, the multinomial logit fails to predict the exact
timing of the crisis that outbreaks in Russia in 1989 and that lasts for the rest of the sample
length (see Table 1). The crisis entry is in fact postponed to 1991 and, most importantly, when
the crisis becomes more severe in 1998, with the default on ruble-denominated Treasury
bonds, the model correctly signals the continuation of the ongoing crisis.
In general, one can conclude that, even in a multinomial logit setting, predicting crisis
entries is a difficult task, more difficult than trying to predict the likelihood of a country
continuing to experience problems, provided it has already entered a crisis. The number of
false alarms, i.e. the number of tranquil periods that are wrongly signalled as crisis, is still
about 36 per cent: in 111 cases out of 308 our model calls for a turbulent period, when in fact
10
The figure for the number of observations correctly called is obtained dividing 415 entries (all crises,
adjustments and tranquil periods) correctly signaled over the overall 585 observations. The figure for the
number of crisis entries correctly called is obtained dividing 28 (crises episodes correctly called) by 37 (the
total number of crises).
- 18 -
a tranquil period materialises thereafter. Out of the 111 cases of false alarm, the majority of
cases can be traced back to crisis entry signals that are not followed by an actual crisis.
Turning to the out-of-sample performance of the model, the multinomial logit seems to
fare well, better than the simple binomial model: the results are shown in Table 9 below.
Table 9 Multinomial Logit Model: out of sample goodness of fit *
True state
Correctly
called
Not correctly
Total
signalled
Pre – crisis
4
1
5
N. of observations correctly called
76%
Post – crisis
17
8
25
N. of crisis entries correctly called
80%
Tranquil
43
11
54
N. of false alarms
20%
Total
64
20
84
* The model is estimated with data up to 1999 to try to predict entries from 2000 onwards
Going back to 1999, we could have been able to anticipate 4 out of 5 crisis episodes
(Peru and Turkey 2000, Argentina 2001 and Uruguay 2002). Using the simple binomial
framework, as seen before, we would have missed the crisis in Peru and, most importantly,
the 2002 crisis in Turkey; in both cases, we would have missed the 2002 crisis in the
Dominican Republic. This sizeable improvement comes to a negligible cost: the number of
observation correctly called is equal to 76 per cent in the multinomial as opposed to 77 per
cent in the binomial and the false alarms are 20 per cent of tranquil observations as opposed
to 18 per cent.
From a practical point of view, the implementation of the model as a regular tool for
policy analysis and decision making is somehow constrained by the existence of publication
lags of macroeconomic data which reduces the nominal forecast horizon to less than one year.
This problem could be easily overcome using Consensus Forecasts of the explanatory
variables or the estimates published by the international financial institutions. A more
demanding solution would be to increase the data frequency, e.g. from annual to quarterly:
this switch, in fact, would also imply a redefinition of the dependent variable, i.e. the timing
of the crisis episodes, which is necessary in order to bring the forecast horizon as close as
possible to the desired one year. (11)
4. EARLY WARNING SYSTEMS USING FINANCIAL MARKET DATA
As already mentioned in the first part of the paper, models based on risky market
instruments that look at how estimated default parameters change through time might shed
some light on the market participants’ perception of sovereign risk and complement the
11
Current research at the Bank of Italy is trying to implement this second alternative.
- 19 -
analysis provided by macroeconomic variables. Following the suggestions from Andritzky
(2003) and Chan Lau (2003), we develop a framework to analyse the behaviour of CDSs
spreads and try to estimate, in such way, the probability of default and the recovery rate
implicit in market data. (12) The results for Argentina, Brazil, Colombia and Mexico are
reported in Chart 2.
12
The details of the model are available from the authors.
- 20 -
Chart 2: CDSs spreads and Default Parameters’ estimates
— Rec. Rate — Prob. ┈┈52 week correlation
— CDS 1 year — CDS 5 years
- 21 -
The left-hand side column of Chart 2 shows spreads on CDSs from January 1998 to
January 2003. Quotes for Colombia are only available starting from September 1998, whereas
data for Argentina are shown until the December 2001 default. We reported the one year
spread, as the focus throughout the paper has been put on the one year forward probability of
default, and the five year spread, as this is usually the most liquid maturity of CDSs
contracts.(13)
The right hand side column of Chart 2 exhibits the estimates for the default parameters,
i.e. the recovery rate and the probability of default, along with the one year correlation
between the two. On average, it seems that during tranquil times the estimated recovery rate
hovers around 80 per cent; during times of distress, instead, it tends to decrease while,
simultaneously, the one year forward probability of default tends to rise. In the case of
Argentina, for example, the recovery rate dropped from around 80 per cent during summer
2001 to almost 20 per cent in November 2001, right before the crisis; at the same time the
probability of default increased from 20 per cent to around 60 per cent. In Brazil the recovery
rate went down from 80 per cent in April 2002 to something between 50 and 60 per cent
during the following months, essentially due to the increased uncertainty determined the
presidential elections, to resume to 80 per cent towards the end of the year. Over the same
period, the probability of default went up from around 20 to 60 per cent. These outcomes are
in line with the results from Andritzky (2003) and Chan Lau (2003): they suggested that,
before a crisis occurs, the correlation between estimated default probabilities and recovery
rates tends to become negative.
The evidence presented here seems to indicate that an early warning system might be
devised on the basis of the analysis of CDSs spreads. One possible way to construct a signal is
to consider instances when the 52 week correlation between the estimated recovery rate and
the one year forward default probability goes below a specific threshold for a prolonged
period of time. As a working assumption, we considered that threshold to be 50 per cent and
we regarded the model to send an early warning signal in the case of a correlation smaller that
the threshold for more than four consecutive weeks; finally a year is dubbed as a crisis year if
that situation happens at least once during the course of the year. Table 10 reports the results
obtained from this rule, in comparison with our original definition of crisis, along with the
signals taken from the binomial and multinomial logit. (14)
13
14
A weakness of the database is that it comprises mid–quotes for CDSs spreads; as a consequence, actual
transactions may have taken place at prices different from the market-makers’ quotes. Generally speaking,
JPMorgan dataset does not give precise information on how liquid the market for CDSs is and therefore on
how reliable the given quotes are.
The choice of the threshold for the correlation and of the number of weeks after which a crisis signal is
warranted, is arbitrary. Choosing a higher threshold and a shorter period of time would lead to a higher
number of crisis correctly called as well as to a higher number of false alarms. This is analogous to the choice
of the threshold for the signal in the binomial and in the multinomial logit model.
- 22 -
Table 10: Crisis signals using CDSs spreads
Country
Year
Argentina
1999
Argentina
2000
Argentina
2001
Argentina
2002
Brazil
1999
Brazil
2000
Brazil
2001
Brazil
2002
Colombia
1999
Colombia
2000
Colombia
2001
Colombia
2002
Mexico
1999
Mexico
2000
Mexico
2001
Mexico
2002
Correctly called
episodes (in %)
Original definition
of crisis
Signal in the
Binomial model
Signal in the
Signal according to
Multinomial model
CDSs spread
0
0
1
1
1
1
1
1
0
0
0
0
0
0
0
0
1
1
1
1
0
1
0
0
1
1
0
0
1
0
0
0
Crisis
Crisis
Crisis
Adjustment
Adjustment
Adjustment
Adjustment
Tranquil
Crisis
Crisis
Crisis
Crisis
Adjustment
Adjustment
Adjustment
Adjustment
0
0
1
1
1
0
0
1
0
0
0
0
1
0
0
1
==
50%
56%
75%
Table 10 shows that CDSs spreads send crisis signals that are in 75 per cent of the cases
in line with the original definition of debt crisis, given in paragraph 4.1. In the sub-sample
considered here, CDSs fare better both than the binomial logit, that correctly marks 50 per
cent of the observations, and the multinomial logit, that reports a score of 56 per cent of
correctly called observations. In two instances, both in Mexico in 1999 and in 2002, CDSs
send a false alarm, that is they signal a crisis that fails to materialise afterwards.
The analysis described so far seems to suggest that a sensible approach to carry out early
warning exercises might entail two steps. In the first place, one should take into account the
behaviour of a set of macroeconomic variables, as they offer a snapshot of the state of health
of an economy and can help to get a first estimate of the likelihood of a crisis. Unfortunately,
in most cases macroeconomic variables can only be obtained once a year and, for most of the
countries, are not always up-to-date, which limits the scope of a timely early warning
exercise. Therefore, as a second step, available market data should be used to obtain up-todate information on market participants views about a particular country.
The main drawback of using risky instruments, especially CDSs, is mainly related to the
scarce liquidity of some emerging financial markets, and thus to the reliability of the
information content that market instruments might carry. This situation limits the scope of
applicability of any EWS based on market data and restricts the number of countries for
which this approach can be applied. That is the reason why we limited our analysis to just
- 23 -
four Latin American countries, for which there exists a sufficiently deep and well functioning
market for CDSs.
5. CONCLUSIONS
The paper focuses on the occurrence of a particular kind of financial crisis, determined by
increasingly unsustainable levels of public and/or external debt, which have grown in
importance in the very recent past, taking over currency crises as a major source of concern in
emerging countries.
After reviewing the relatively small academic literature on early warning systems for debt
crises, we have drawn the conclusion that these episodes may take a variety of different
forms, which range from outright default to more general debt-servicing difficulties. This
consideration has led us to construct a debt crisis indicator able to account for such different
forms. This indicator has been used to evaluate a series of debt crisis episodes experienced, in
the period 1980-2002, by a relatively large sample of emerging market economies,
characterised by significant access to international capital markets.
The next step was to discover which macroeconomic factors might be at the roots of a
debt crisis. We have first applied a classic binomial logit approach, according to which the
variables that are significant in explaining debt crisis episodes are mainly those that measure
the burden of external indebtedness and foreign-currency generating capabilities of a country.
We then improve upon the out-of-sample predictive ability, the most important aspect one has
to take into account in order to evaluate an early warning model, performing a multinomial
logit analysis, characterised by the existence of three, rather than two, regimes. The main flaw
of both the previous specifications is the high number of false alarms launched.
As a further step, we try to integrate the analysis based on macroeconomic variables with
the approach based on risky market instruments. A model that combines various technical
aspects from the existing theoretical literature has been developed in order to estimate the one
year forward probability of default and the recovery rate from CDSs quotes for Argentina,
Brazil, Colombia and Mexico. We subsequently have worked out a simple rule to obtain crisis
signals from the default parameters that produced better results, in terms of the percentage of
episodes correctly identified and in terms of false alarms, with respect to the binomial and
multinomial logit approach.
We concluded suggesting that an ideal EWS should be able to integrate the two aspects.
In this respect, timely information on the relevant macroeconomic variables are needed along
with reliable and robust market data. With more frequent macroeconomic data, and a wider
set of countries for which financial data are available, further research could be carried out
with the aim of better integrating the two aspects of an EWS for debt crises.
- 24 -
References
Andritzky, J., 2003, “Implied Default Probabilities and Default Recovery Ratios: An Analysis
of the Argentine Crisis 2000 - 2002”, University of St. Gallen, available via the internet at
http://www.vfs.unizh.ch/papers.htm.
Berg A., Borensztein E. and C. Pattillo, 2004, “Assessing Early Warning Systems: How Have
They Worked in Practice?”, IMF Working Paper, WP 04/52.
Bussiere M., and M. Fratzscher, 2002, “Towards a New Early Warning System of Financial
Crises”, European Central Bank Working Paper 145.
Chan-Lau J.A., 2003, “Anticipating Credit Events Using Credit Default Swaps, with an
Application to Sovereign Debt Crises”, IMF Working Paper, WP 03/106.
Ciarlone, A., Trebeschi, G., 2004, “Currency and Debt Crises: A Review of The Early
Warning Systems”, in S. Wilkin, eds., Country and Political Risk: Practical Insight for
Global Finance (Risk Book).
Detragiache E., and A. Spilimbergo, 2001, “Crises and Liquidity: Evidence and
Interpretation”, IMF Working Paper, WP 01/2.
Frankel J. A., and A. K. Rose, 1996, “Currency Crashes in Emerging Markets: an Empirical
Treatment”, Journal of International Economics 41, 351-366.
Kaminsky G. L., Lizondo S. and C. M. Reinhart, 1998, “Leading indicators of currency
crises”, IMF Staff Papers, Vol. 88 No. 2, 1-48.
IMF, 2002, “Early Warning System Models: The Next Step Forward”, Global Financial
Stability Report, March.
Manasse P., Roubini N., and A. Schimmelpfennig, 2003, “Predicting Sovereign Debt Crises”,
IMF Working Paper, WP 03/221.
Pescatori A., and A. N. R. Sy, 2003, “Debt Crises and the Development of International
Capital Markets”, IMF Working Paper, WP 04/44.
Sy A. N. R., 2003, “Rating the Rating Agencies: Anticipating Currency Crises or Debt
Crises?”, IMF Working Paper, WP 03/122.
Visco I., 1978, “On obtaining the right sign of a coefficient estimate by omitting a variable
from the regression”, Journal of Econometrics 7, 115-117.
- 25 -