Russian banks sovereign ratings:
a comparative study
S.Smirnov, A. Kosyanenko, V. Naumenko,
V. Lapshin, E. Bogatyreva
Higher School of Economics, Moscow
Introduction
The aim of the research was to assess credit ratings’ quality for
the purpose of Bank Counterparties Credit Risk Assessment, in
order to use them in credit risk models under IRB approach.
Presentation plan:
General requirements to external credit ratings
Properties of migration matrices
Assessment of conditional intervals for PD
Entropy measures
Mapping to model-based PD
Conclusions
2
Using credit ratings in models
Probability of default (PD) is one of the
major building blocks in credit risk
management.
External credit rating can be used as an input
variable in PD-models. Using credit ratings
in addition to other sources of information
about borrower's credit risk (e.g. financial
ratios, market-based information) may
improve the prediction power of credit risk
models (see [Kealhofer, 2003], [Loffler,
2007])
By using credit ratings as input to credit risk
models one should assess the uncertainty of
these variables (see [Basel II]).
External
rating
PD
Capital
requirements,
Pricing,
Provisions
3
Basel II requirements for ratings
Under the IRB approach different exposures should be treated separately, e.g.
corporate, sovereign, bank, retail, and equity (Basel II, §215).
“Irrespective of whether a bank is using external, internal, or pooled data
sources, or a combination of the three, for its PD estimation, the length of the
underlying historical observation period used must be at least five years for at
least one source. If the available observation period spans a longer period for
any source, and this data are relevant and material, this longer period must be
used” (Basel II, § 463)
Ratings assigned by for external credit assessment institution (ECAI) should
be recognized by regulator and satisfy 6 eligibility criteria: Objectivity,
Independence, International access/Transparency, Disclosure, Resources,
Credibility (Basel II, § 91).
:
4
National rating agencies in Russia
There are four largest national rating agencies recognized by the
Russian Central Bank: RusRating, Expert RA, National rating
agency (NRA) and AK&M.
Historical data for the purpose of the research was provided by
RusRating and AK&M. Information about ratings assigned by
NRA and Expert RA was taken from their web-sites.
Rating data contains monthly information about rating assigned
to Russian banks from January, 2001 to May, 2010.
5
Dynamics of rating assignment
As of May 1, 2010
there
were
the
following numbers of
efficient credit ratings
in the banking sector:
RusRating – 51
AK&M – 33
NRA – 65
Expert RA - 113
In October 2008 Russian Central Bank recognized ratings assigned by national
rating agencies (RusRating, Expert RA, NRA and AK&M) for the purpose of
granting unsecured loans.
6
Rating Transitions
Rating agencies are not likely to revise their ratings: since 2001 there were
only few rating downgrades. The number of rating upgrades is much more
substantial.
Total
Since 01.01.2008
Down
grades
Up
grades
Obs
No.
Down
grades
Up
grades
Obs
No.
RusRating
11
85
3375
2
34
1315
Expert RA
13
18
2365
13
18
2261
NRA
0
19
922
0
19
888
AK&M
1
4
611
1
4
601
There are several cases when ratings withdrawals due to termination of the
contract followed rating downgrades in a month or two. In the beginning of
2010 Expert RA had 5 such facts.
7
Rating philosophies
• Ratings Point in Time indicate the current
probability of issuer’s default. They are likely
to change significantly during the bad times.
• Ratings Through the Cycle indicate the
average probability of default during the long
period of time. They are not likely to change
during the economic cycle. However it’s likely
that default probabilities associated with
ratings grades do change.
Transition matrix computation
• Cohort approach takes into account only the
initial and terminal states of the institution in
question.
• Duration approach takes into account time
spent in every rating grade.
• Under conditions of first order Markov
process, time homogenous matrix structure for
Russian agencies these approaches coincide.
Migration matrices: the case of S&P
Typical S&P migration matrix (expressed in monthly transition probabilities
for the purpose of comparison with Russian rating agencies):
Source: S&P 2008 Global Corporate Default Study and Rating Transitions
Key features are:
Distinct diagonal line (taking into account aggregation of rating classes).
Existence of non-diagonal elements which gives evidence of rating upgrades
and downgrades.
10
Migration matrices: RusRating
Data on rating transitions had monthly frequency. Each number in a diagonal
cell of a migration matrix shows probability of the fact that rating will not
change in a month period of time. Numbers below the diagonal line show the
probability of rating upgrades, above –downgrades.
A+
A
ABBB+
BBB
BBBBB+
BB
BBB+
B
BCCC+
CCC
CCC-
A+
A
ABBB+ BBB
BBB- BB+
BB
BBB+
B
BCCC+ CCC
1,000
0
0
0
0
0
0
0
0
0
0
0
0
0
0 1,000
0
0
0
0
0
0
0
0
0
0
0
0
0
0 1,000
0
0
0
0
0
0
0
0
0
0
0
0
0
0 0,988 0,012
0
0
0
0
0
0
0
0
0
0
0
0 0,013 0,981 0,006
0
0
0
0
0
0
0
0
0,003
0
0 0,003 0,011 0,980 0,003
0
0
0
0
0
0
0
0
0
0
0
0 0,019 0,977 0,002
0
0 0,002
0
0
0
0
0
0
0 0,003
0 0,028 0,969
0
0
0
0
0
0
0
0
0
0
0 0,003 0,004 0,013 0,975 0,004
0
0
0
0
0
0
0
0
0 0,002
0
0 0,036 0,959 0,002
0
0
0
0
0
0
0
0
0
0 0,003 0,007 0,030 0,956
0 0,003
0
0
0
0
0
0
0
0
0 0,007
0 0,035 0,950 0,007
0
0
0
0
0
0
0
0
0
0
0
0 0,063 0,937
0
0
0
0
0
0
0
0
0
0
0
0
0 0,111 0,889
0
0
0
0
0
0
0
0
0
0
0
0
0
0
RusRating migration matrix demonstrates sufficient amount of both upgrades
and downgrades.
11
Migration matrices: Expert RA, AK&M
Migration matrices for Expert RA and AK&M have much less non-diagonal
elements than migration matrix for RusRating.
Expert RA
A++
A+
A
B++
B+
B
C++
C+
C
D
A++
A+
A
B++
B+
B
C++
C+
1,000
0
0
0
0
0
0
0 1,000
0
0
0
0
0
0 0,003 0,982 0,013
0
0
0
0
0 0,004 0,975 0,006
0
0
0
0
0 0,026 0,930
0
0
0
0
0
0 0,033 0,945
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
C
D
0
0
0
0
0
0
0
0
0 0,003
0 0,011
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
AK&M
A++
A++
A+
A
B++
B+
B
C++
C+
C
D
A+
A
A0
0
0
0 0,975
0
0
0 0,974
0
0 0,007
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
B++
B+
B
C++
C+
0
0
0
0
0
0
0
0
0
0
0 0,005
0
0
0
0 0,975
0
0
0
0 0,042 0,896
0
0
0
0
0 0,909
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
C
0
0
0
0
0
0
0
0
0
0
D
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
12
Migration matrices: NRA
Rating history of NRA has almost no downgrades.
AAA AA+
AA
AAA+
A
ABBB+ BBB
BBB- BB+
BB
BBD
AAA
0,949
0
0 0,026
0
0
0
0
0
0
0
0
0
0
AA+
0,023 0,977
0
0
0
0
0
0
0
0
0
0
0
0
AA
0
0 0,964
0
0
0
0
0
0
0
0
0
0
0
AA0
0
0 1,000
0
0
0
0
0
0
0
0
0
0
A+
0
0
0 0,008 0,992
0
0
0
0
0
0
0
0
0
A
0
0
0
0 0,022 0,978
0
0
0
0
0
0
0
0
A0
0
0
0
0 0,029 0,968
0
0
0
0
0
0
0
BBB+
0
0
0
0
0
0 0,047 0,922
0
0
0
0
0
0
BBB
0
0
0
0
0
0 0,081
0 0,919
0
0
0
0
0
BBB0
0
0
0
0
0
0
0
0 1,000
0
0
0
0
BB+
0
0
0
0
0
0
0
0
0 0,042 0,958
0
0
0
BB
0
0
0
0
0
0
0
0
0
0
0 0,800
0 0,200
BB0
0
0
0
0
0
0
0
0
0
0
0 0,909
0
13
Confidence intervals
Confidence interval is an interval with lower (L) and upper (U)
bounds that covers the unknown true parameter, i.e. L < p < U
with some predefined probability:
Prob{L < p < U} = 1 − α.
Confidence intervals is a standard industry tool to assess
uncertainty of PD estimations (see, for example [OeNB, 2004]).
One of the major factors that influence the length of confidence
intervals for PD is the amount of data available. There are
research papers that show that to some extent it is impossible to
statistically distinguish investment grade rating classes (see
[Lawrenz, 2008]).
14
Confidence interval methodology
To calculate confidence intervals for PD one should:
Fit a priori unconditional PD distribution from external data
(Russian Deposit Insurance Agency PD model) as Beta
distribution – very good agreement. Estimated parameters (a,b) =
(1,16.3).
Regard each month for each bank with given rating as a trial:
success if no default, failure if default. Form posterior
distribution for PD: Beta (number of defaults + 1, number of nondefaults + 16.3).
Find 95% confidence interval for Beta distribution with
estimated parameters and plot together with (number of
observations)-1.
15
Confidence intervals: RusRating
16
Confidence intervals: Expert RA
17
Confidence intervals: NRA
18
Confidence intervals: AK&M
19
Conditional entropy
Conditional entropy measures new information (in bits) contained in each
successive rating value (randomly selected).
Conditional entropy
Given migration matrix pi,j and
unconditional
probabilities
pi
(expected) conditional entropy is
0.2
0.15
0.1
n
n
 pi  pi , j log 2 pi , j
i 1
j 1
0.05
0
RR
AK&M
NRA
Expert
To understand what happened to credit quality of the rating object (3
possibilities: whether it improved, deteriorated or remained the same) it is
necessary to obtain data over the period of (months):
RusRating – 9; NRA– 11; Expert RA– 13; AK&M – 21.
20
Mapping to model assessed PD
Ratings were mapped to PD estimates derived from econometric
model based on balance sheet data. This model is used by
Deposit Insurance Agency to assess PD of banks –participants of
Deposit Insurance System.
The following measures were calculated in order to estimate the
accuracy of ratings:
 average PD for each rating grade;
confidence intervals for PD according to each rating grade;
probability that PD associated with different rating grades will
coincide.
21
Rating grades comparison methodology
Given PD samples for 2 different rating values, test a
hypothesis: “these 2 samples really come from the same PD
distribution”.
Non-parametric Kolmogorov-Smirnov test using
max CDF1 ( x)  CDF2 ( x)
x
as test statistics.
Enter the p-value for each pair of rating values (including
general population) in a table.
22
Mapping: RusRating
Probability of PD coincidence
All
AAA
BBB+
BBB
BBBBB+
BB
BBB+
B
BCCC+
CCC
All
-
AAA
BBB+
58%
58% 56%
1%
0%
0%
11%
1%
1%
51%
0%
1%
0%
92% 4%
27%
53%
91%
38%
16%
79%
0%
5%
0%
BBB
56%
92%
2% 37%
41%
47%
25%
8%
71%
0%
9%
0%
BBB1%
4%
2%
0% 0%
0%
4%
1%
1%
2%
15%
0%
BB+
0%
27%
37%
0%
65% 3%
0%
0%
10%
0%
0%
0%
BB
0%
53%
41%
0%
65%
9% 0%
0%
2%
0%
0%
0%
BB11%
91%
47%
0%
3%
9%
0% 0%
15%
0%
0%
0%
B+
1%
38%
25%
4%
0%
0%
0%
5% 3%
0%
20%
0%
B
1%
16%
8%
1%
0%
0%
0%
5%
3% 0%
51%
0%
B51%
79%
71%
1%
10%
2%
15%
3%
3%
0% 1%
0%
CCC+
0%
0%
0%
2%
0%
0%
0%
0%
0%
0%
20% 0%
CCC
1%
5%
9%
15%
0%
0%
0%
20%
51%
1%
20%
0% -
23
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
Mapping: Expert RA
Probability of PD coincidence
All
All
A++
A+
A
B++
B+
B
A++
100%
0%
0%
4%
12%
6%
6%
A+
0%
100%
3%
0%
0%
0%
0%
A
0%
3%
100%
0%
0%
0%
0%
B++
4%
0%
0%
100%
1%
0%
0%
B+
12%
0%
0%
1%
100%
34%
34%
B
6%
0%
0%
0%
34%
100%
100%
6%
0%
0%
0%
34%
100%
100%
24
Mapping: NRA
Probability of PD coincidence
All
AAA
AA+
AA
A+
A
ABBB+
BBB
BB+
BB
BB-
All
-
AAA
AA+
13%
13% 25%
0%
24%
9%
0%
17%
34%
1%
93%
6%
32% 0%
8%
2%
0%
2%
17%
1%
82%
4%
AA
25%
32%
0% 5%
28%
3%
11%
51%
12%
72%
51%
A+
0%
0%
0%
0% 1%
3%
0%
0%
0%
36%
0%
A
24%
8%
5%
0%
5% 1%
38%
52%
0%
59%
5%
A9%
2%
28%
1%
5%
5% 3%
15%
4%
100%
25%
BBB+
0%
0%
3%
3%
1%
5%
3% 13%
0%
67%
3%
BBB
17%
2%
11%
0%
38%
3%
3%
38% 0%
70%
3%
BB+
34%
17%
51%
0%
52%
15%
13%
38%
5% 74%
30%
BB
1%
1%
12%
0%
0%
4%
0%
0%
5%
11% 74%
BB93%
82%
72%
36%
59%
100%
67%
70%
74%
11%
38% -
25
6%
4%
51%
0%
5%
25%
3%
3%
30%
74%
38%
Mapping: AK&M
Probability of PD coincidence
All
A+
A
B++
B+
B
All
-
A+
A
29%
29% 0%
47%
3%
5%
19% 43%
4%
1%
B++
0%
19%
0% 0%
1%
B+
47%
43%
0%
37% 6%
B
3%
4%
0%
37%
5%
1%
1%
6%
27%
27% -
26
Conclusions

It is reasonable to use external credit rating as an input parameter in credit
risk models. Accuracy of these rating assessment should be taken into
account.

However according to our findings we can not recommend to use ratings
assigned by national rating agencies in credit risk models as the only
source of information due to the lack of credibility:


rating are not likely to be downgraded;

sometimes there is no uniform dependence between rating grades and
PD;

in most cases we can not differentiate between rating grades.
When new data will be accumulated it will be possible to estimate rating
accuracy once more and probably use ratings as an alternative source of
credit quality information.
27
References
1.
2.
3.
4.
5.
Basel Committee on Banking Supervision. International
Convergence of Capital Measurement and Capital Standards. A
Revised Framework. Bank for International Settlements. June
2006 (Basel II).
Kealhofer, 2003. Quantifying Credit Risk I: Default Prediction.
Finandal Analysts Journal, 59, pp. 30-44.
Loffler, 2007. The Complementary Nature of Ratings and MarketBased Measures of Default Risk. The Journal of Fixed income,
pp. 38-47.
OeNB (Oesterreichische Nationalbank), 2004. Rating Models and
Validation in Guidelines on Credit Risk Management.
Lawrenz J. Assessing the estimation uncertainty of default
probabilities.// Kredit und Kapital. -2008.-Vol. 41 (2). pp. 217238.
28
Thank you for your attention!
29