Unofficial translation
BOARD OF THE BANK OF LITHUANIA
RESOLUTION No 140
of 9 November 2006
ON THE REGULATIONS ON VALIDATION AND ITS ASSESSMENT
Vilnius
(Valstybės žinios (Official Gazette) No 142-5444, 2006)
Acting in observance of Article 9 Law of the Republic of Lithuania on the Bank of
Lithuania (Valstybės žinios (Official Gazette) No 99-1957, 1994; No 28-890, 2001) and in
implementing Directive 2006/48/EC of the European Parliament and of the Council of 14 June
2006 relating to the taking up and pursuit of the business of credit institutions (recast) (OJ 2006 L
177, p. 1), the Board of the Bank of Lithuania has r e s o l v e d:
1. To approve the Regulations on Validation and Its Assessment (attached).
2. To establish that the present Resolution comes into effect as from 1 January 2007.
Chairman of the Board
Reinoldijus Šarkinas
2
APPROVED
by Resolution No140
of the Board
of the Bank of Lithuania
of 9 November 2006
REGULATIONS ON VALIDATION AND ITS ASSESSMENT
(Valstybės žinios (Official Gazette) No 142-5444, 2006)
CHAPTER I
GENERAL REGULATIONS
1.
Regulations on Validation and Its Assessment (hereinafter – the Regulations) shall
apply to banks holding the license issued by the Bank of Lithuania and to the Central Credit
Union, which for their capital adequacy calculation purposes apply the Internal ratings Based
Approach (hereinafter – IRB approach) and (or) the operational risk Advanced Measurement
Approach (hereinafter – AMA).
2.
The purpose of the present Regulations is to define the standards for validation of
the IRB approach and AMA and for assessing such validation.
3.
The present Regulations have been developed in observance of Directive
2006/48/EC of the European Parliament and of the Council of 14 June 2006 relating to the taking
up and pursuit of the business of credit institutions (recast) (OJ 2006 L 177, p. 1) and on the basis
of principles set forth in the following documents:
3.1.
Guidelines on the implementation, validation and assessment of Advanced
Measurement (AMA) and Internal Ratings Based (IRB) Approaches issued by the Committee of
European Banking Supervisors;
3.2.
Studies on the Validation of internal rating systems, Validation of low default
portfolios in the Basel II framework and The treatment of expected losses by banks using the
AMA under the Basel II framework issued by the Basel Committee on Banking Supervision;
3.3.
Internal ratings–based systems for corporate credit and operational risk advanced
measurement approaches for regulatory capital issued by the Federal Deposit Insurance
Corporation.
4.
Requirements of the present Regulation shall apply together with the requirements
of the General Regulations for the Calculation of Capital Adequacy approved by Bank of
Lithuania Board Resolution No. 138 of 9 November 2006 (Valstybės žinios (Official Gazette)
No 142-5442, 2006).
CHAPTER II
VALIDATION OF THE IRB APPROACH
5.
Terms used for the purpose of this Chapter:
5.1.
Validation encompasses the range of processes and activities aimed at assessing
the credibility of the rating system structure and assignment of ratings, as well as of the
quantification of risk parameters and processes.
5.2.
Back-testing means validation method that involves comparing of estimated risk
parameters with respective realised values.
5.3.
Benchmarking means validation method that involves comparing internal ratings
and estimated risk parameters with respective internal and external ratings and parameters
obtained using other estimation techniques.
2
3
5.4.
Low-default portfolio means portfolio with more than one case of realised default
event, or portfolio free from any cases of realised defaults.
5.5.
Risk parameters means quantitative outcomes of the rating system and
quantification methods , i.e. probability of default (hereinafter – PD), loss given default
(hereinafter – LGD), conversion factor (hereinafter – CF), expected loss (hereinafter – EL), etc.
5.6.
Obligor grade means a risk category within a rating system's obligor rating scale,
to which obligors are assigned on the basis of a specified and distinct set of rating criteria and from
which estimates of PD are derived.
5.7.
Facility grade means a risk category within a rating system's facility rating scale,
to which exposures are assigned on the basis of a specified and distinct set of rating standards and
from which own estimates of LGDs and (or) CFs are derived.
5.8.
Assignment process means associating the obligor (exposure) with the respective
grade of the rating scale of the borrower (exposure), or in case of retail exposures - with respective
risk pools in accordance with the assignment criteria.
5.9.
Discriminatory (separation) power means ability of the rating system to
distinguish potential defaulting borrowers from non-defaulting ones, or in case of retail exposures
– to distinguish exposures which will be defaulted from those which will not be defaulted.
5.10. Calibration means determining risk parameters for obligor or facility grades and
risk pools.
5.11. Quantification process involves collection of data, estimation of risk parameters,
mapping such parameters to respective obligor or facility grades or risk pools and using them for
the calculation of capital adequacy.
5.12. Confidence level means range between two values of the measure being assessed
to which the latter belongs under a certain probability.
5.13. Through–the–cycle rating is assigned when the obligor or exposure risk is
assessed relying on assumption of the existing economic downturn phase.
5.14. Point–in–time rating is assigned when the obligor or exposure risk is assessed on
the basis of current situation and existing phase of the economic cycle.
5.15. Rating transition matrix shows changes in the bank’s rating system grades or risk
pools of a given period, i.e. percentage of obligors (exposures) previously assigned to a particular
grade or risk pool, which is presently transferred to any other grade or risk pool.
5.16. Internal models method means method applied for the calculation of capital
required for covering credit risk of equities.
5.17. External vendor model means credit assessment model developed by third parties
used for the bank's risk assessment and management purposes.
5.18. Reference data set means data set used for the estimation of the respective risk
parameter.
SECTION I
VALIDATION PROCESS
6.
The bank shall have in place the adequate validation process of its IRB approach,
i.e. covering all elements listed in Annex 1 and contributing to consistent and meaningful
validation of the internal rating systems and processes and to determining the conformity of these
systems and processes with the requirements of the present Regulations and of Part II, Chapter IV
of the General Regulations for the Calculation of Capital Adequacy. Requirements of the present
Regulations shall apply to the extent of their appropriateness in each particular case, e.g., when the
bank applies the slotting criteria approach to its specialised lending exposures, the quantitative
validation of risk parameters shall not be required.
7.
The bank’s validation shall encompass qualitative elements (assessment of structure
of the rating system, data quality, use of rating systems in other activities (i.e., not only for the
capital adequacy calculation purposes, etc.) and quantitative elements (discriminatory power,
accuracy of calibration , assessment of stability, etc.). The bank shall select the most appropriate
3
4
validation methods on its own discretion. When quantitative validation performed is not reliable
because of the insufficient amount of information, the bank shall pay more attention to the
qualitative validation.
8.
Validation shall be an ongoing and regular process. Validation methods and data
shall be used in consistent manner. The bank shall review on a regular basis its validation methods
and data used in consideration of changes in market or operating conditions (e.g., upon change of
the extent of data applied in assignment or quantification processes , duration of historical data
observation periods, characteristics of exposures or obligors , crediting standards, etc.).
9.
The bank’s validation process and its outcomes shall be subject to review by the
respective employees of the bank independent from staff in charge of the development and
implementation of validation processes. An independent review may be carried out by one or more
structural units. The Internal Audit Committee shall monitor the enforcement of the developed
validation processes.
10.
The process of validation and all its elements (applied methods, their justification,
used data, liability, accountability, independence, scope, documentation, regularity, outcomes and
actions in consideration of obtained results), changes in such elements and their reasons shall be
documented. The documentation shall be subject to regular reviews and amended where
appropriate. The process of validation and all its elements shall be approved by a respective bank
or bank group management body, structural unit or committee.
11.
The bank shall establish the general principles, what actions shall be taken on the
basis of validation outcomes. The bank shall have in place reliable internal standards for such
situations when deviations of realised risk parameters from forecasted quantitative risk parameters
become quite significant. Where practicable, the bank shall with due regard to the specifics of the
applicable quantitative validation methods establish the acceptable intervals of the results of
methods used for determining the discriminatory power, accuracy of calibration and stability as
well as of other quantitative validation methods, also providing for the respective actions when
resulting values do not fall within this interval of acceptable estimates. The bank shall take into
account the economic cycles and other volatility of risk parameters of systemic nature. The bank
shall to the extent possible provide for the situations, when these intervals and respective actions
may be changed.
12.
The Bank of Lithuania shall carry out regular reviews of the bank’s validation
process and separate elements thereof (validation methods applied by the bank, their justification,
used data, regularity, liability, accountability, independence, scope, documentation, outcomes and
actions in consideration of obtained results), in order to assess whether the bank’s validation
process is consistent with requirements of the present Regulations, the internal rating systems and
processes used by the bank are reliable and in line with the requirements set forth under Part II,
Chapter IV of the General Regulations for the Calculation of Capital Adequacy.
SECTION II
VALIDATION OF RATING SYSTEMS
13.
For the purpose of validation or rating systems (and applicable methods) the bank
shall observe the principles of objectivity, accuracy, stability, monotony, appropriateness,
conservatism and consistency:
13.1. in observance of the principle of objectivity, the bank shall guarantee that obligors
(exposures) assigned to same obligor (facility) grade or risk pool have similar characteristics and
level of risk. For the purpose of development of rating systems, assignment of obligors
(exposures) to grades or risk pools and quantification processes, the bank shall take into account
expert judgements, including the overrides. The bank shall consider how the expert judgement is
applied in order to obtain objective outcomes of the rating system. When the bank relies
exclusively upon expert judgement when assigning obligors or exposures to grades, the assignment
standards shall be formulated in clear manner so as to avoid any interpretations thereof, and the
bank shall have in place the assessment guide developed in appropriate manner;
4
5
13.2. in observance of the principle of accuracy, the bank shall guarantee that the
assessment of creditworthiness of the obligor (exposure) is accurate, quantified risk parameters
conform to the respective realised parameters and input values are appropriate;
13.3. in observance of the principle of stability, the bank shall guarantee that ratings and
risk parameters will not change, when the underlying risk characteristics of obligors or exposures
remain the same (excluding changes of ratings and risk parameters related with the developments
of the economic cycle);
13.4. in observance of the principle of monotony, the bank shall guarantee that lowergrade risk parameter values will be more conservative than the respective risk parameter values of
the higher grade or risk group;
13.5. in observance of the principle of appropriateness, the bank shall guarantee that all
information relating to the obligor (exposure) will be assessed, e.g., whether the bank uses
sufficient amount of assignment criteria for the purpose of assignment obligors to respective
grades relying on expert judgements. The bank, which applies statistical models and, e.g., does not
assess the measure of profitability, liquidity, turnover, debt servicing or any other significant
criterion, shall have to prove that such criterion is insignificant;
13.6. in observance of the principle of conservatism, the bank shall guarantee that ratings
will be assigned in conservative manner, and risk parameters will be subject to the conservatism
margin in observance of the expected range of estimation errors;
13.7. in observance of the principle of consistency, the bank shall guarantee that rating
systems and respective applicable methods are conceptually reasonable. For example, upon
increase of ceteris paribus profitability, the obligor’s condition should not be considered as
worsening.
14.
The bank applying several rating systems with different characteristics shall
guarantee that such systems are applied in consistent manner and that it is well-aware of their
differences, including the ability to match, where appropriate, the outcomes of different rating
systems.
SECTION III
VALIDATION OF RATING SYSTEM STRUCTURE AND ASSIGNMENT PROCESS
15.
The bank shall carry out regular assessment of the rating system structure and the
process of assignment of obligors or exposures to grades or risk pools, with a view to determining
their consistency with the requirements set forth under Part II, Chapter IV of the General
Regulations for the Calculation of Capital Adequacy.
1. Assessment of the rating system structure
16.
The bank shall assess:
16.1. whether in case of applying several rating systems to different obligor or exposure
categories the assignment of obligors or exposures to the particular rating system is appropriate
and whether such assignment criteria are appropriate and subject to regular review;
16.2. whether the rating system covers a separate obligor rating scale (for exposure
classes of central governments and central banks, institutions and corporates) and - where own
estimates of LGD and (or) CF are used – the bank’s own facility rating scale;
16.3. in consideration of characteristics of the respective exposure class or subclass, the
adequacy of granularity per each rating scale, possible concentration of obligors or exposures in
grades or risk pools, etc.;
16.4. whether there’s sufficient number of obligors or exposures in each grade or risk
pool for the quantification and validation of risk parameters;
16.5. whether obligors or exposures with the same risk features are assigned to the same
grades and risk pools;
5
6
16.6. whether the rating system structure is in line with the requirements of Subsection
9.1.1, Part II, Chapter IV of the General Regulations for the Calculation of Capital Adequacy.
2. Validation of assignment process
17.
The bank shall asses the validity of methods (expert judgements, models (statistical
models, models derived by experts) and (or) hybrid methods) applied when assigning obligors or
exposures to grades or risk pools.
18.
The bank shall carry out ex ante and ex post validation of methods used in the
assignment process.
19.
The bank shall determine the appropriateness of the assignment criteria , for
example, whether:
19.1. the criteria are matched with the bank’s internal crediting standards and the bank’s
policy when there are problem credits;
19.2. the discriminatory power of the individual assignment criterion is adequate, i.e.
whether in observance of such criterion defaulting obligors can be distinguished from the nondefaulting ones;
19.3. the discriminatory power of the individual criterion has reduced and, if so, then
what are the reasons for such reduction;
19.4. the individual assignment criterion can be replaced by another criterion;
19.5. the individual criterion or all criteria applicable in the process of assignment to
grades or risk pools can be replaced by external data;
19.6. there’re systemic changes of input parameters or assignment criteria , if so, what are
such changes;
19.7. the selected period is sufficient for the assessment of criteria.
20.
The bank shall assess whether the respective internal documents of the bank have
been observed during the assignment process.
21.
The bank shall assess whether used definitions of the obligor (facility) grades and
risk pools conform to the requirements provided for in Subsection 9.1.1, Part II, Chapter IV of the
General Regulations for the Calculation of Capital Adequacy.
22.
The bank shall assess the information about the obligor (exposure) used in the
assignment process, i.e. determine, whether the information about obligor (exposure) used in the
process of assignment of obligors (exposures) to grades or risk pools satisfies the requirements of
Subsection 9.1.1, Part II, Chapter IV of the General Regulations for the Calculation of Capital
Adequacy.
23.
The bank which uses a statistical model for the purpose of assigning obligors
(exposures) to grades or risk pools, shall:
23.1. assess methodology used for the model development;
23.2. determine whether data are comprehensive, of good quality, accurate, appropriate
and representative;
23.3. determine the appropriateness of input parameters. Where the bank uses regressive
models for the purpose of determining the appropriateness of input parameters in developing such
models, it shall assess the outliers, economic logic of input parameters and their statistical
significance:
23.3.1. to determine the economic logic of input parameters, the bank shall assess the
validity of a plus or minus sign of the input parameter. For example, when developing the logistic
regression credit assessment model, the bank uses profitability ratio as one of the input parameters,
but in the created regressive equation the profitability ratio parameter has a plus sign, i.e., upon
increase of the obligor’s profitability, the equation result (i.e., PD) would increase, which is
incompatible with economic logic. In such case this parameter should be refused;
23.3.2. to determine statistical significance of input parameters, the bank may apply
different statistical values (t-values, F-values, etc.). Additionally, the bank shall assess the degree
of multicolinearity of the input parameters, i.e. correlation between input parameters;
6
7
23.4. carry out model testing with data excluded from data sample used for model
development, i.e. use out-of-time and out-of-sample data.
24.
In case of hybrid models, the bank shall assess the significance of expert judgement
in the assignment process. For example, the bank may compare default rates of categories of
obligors which at the initial stage of the process of assignment to grades or risk pools had equal
ratings, but after expert assessment received different ratings, with the respective PDs established
for such categories. Similarly, the bank may compare default rates of categories of obligors which
at the initial stage of the process of assignment had different ratings, but after expert assessment
were rated on equal grounds, with respective PDs established for such categories.
25.
The bank shall determine whether the requirements of Subsections 9.1.2, 9.1.3 and
9.2.3 Part II, Chapter IV of the General Regulations for the Calculation of Capital Adequacy are
observed.
26.
The bank shall evaluate the override process. The bank shall define and document
the acceptable limits of the difference between the obligor or facility grade assigned during the
process of assignment and the newly established rating. The bank shall apply and document the
monitoring system for overrides. The bank shall regularly analyse the characteristics of exposures
the ratings assigned to which had been overridden per each person responsible for such overrides.
The bank shall assess the observance of requirements set forth under Subsection 9.1.2, Part II,
Chapter IV of the General Regulations for the Calculation of Capital Adequacy.
27.
The bank shall regularly assess the discriminatory power of rating systems and
applicable methods. The bank may apply the methods referred to in Annex 2 hereof or other
appropriate methods. The bank shall:
27.1. determine whether the outcomes of applicable discriminatory power determination
methods fall within the acceptable interval of values established in advance, and if such outcomes
do not fall within the aforementioned interval, the bank shall determine the reasons for that;
27.2. rely on actual data (including data of the bank operating in the Republic of
Lithuania) covering the maximum possible period;
27.3. take into account the rating system philosophy (i.e. use of through-the-cycle or
point-in-time ratings) which was followed in developing the rating system. The bank which uses
rating systems based on different rating philosophy shall consider all differences of these systems
when assessing the discriminatory power.
28.
The bank shall carry out regular assessments of rating transitions during the
respective period, e.g.:
28.1. determine whether diagonal values of the rating transition matrix exceed other
values of the same row, and if not, specify the reasons for that;
28.2. assess the monotony of the rating transition matrix, where the matrix is not
monotonous, identify possible reasons;
28.3. determine whether rating transitions are of systemic nature, i.e. whether the values
at the top of the rating transition matrix exceed the bottom values of the matrix or vice versa, and
possible reasons for such transitions (e.g., changes of the bank strategy and crediting standards).
29.
The bank shall perform regular analysis of the sensitivity of rating systems
(including stress tests), identify changes in grades or risk pools according to the applicable stress
testing scenarios. The bank shall assess the observance of requirements set forth under Subsection
9.1.6, Part II, Chapter IV of the General Regulations for the Calculation of Capital Adequacy.
30.
The bank shall compare on regular basis its internal ratings with respective external
or internal benchmarks, e.g., ratings assigned by external credit risk assessment institutions (except
where it is impossible to find benchmarks suitable for comparison). The bank may:
30.1. perform regular reassignments of obligors or exposures to grades or risk pools, e.g.:
30.1.1. the process of assignment of obligors or exposures to grades or risk pools carried
out by experts may be repeated, comparing the newly assigned ratings with the old ones;
30.1.2. a statistical model may be applied when assigning to grades or risk pools the
obligors or exposures which had been previously assigned to grades or risk pools on the basis of
expert judgement;
7
8
30.1.3. obligors or exposures which had been previously assigned to grades or risk pools
using statistical model may be assigned to grades or risk pools by experts;
30.2. compare transitions of internal and external ratings;
30.3. assess the stability of the rating system, e.g.:
30.3.1. determine the impact of the obligors’ structure changes on the rating system’s
outcomes;
30.3.2. analyse characteristics of obligors or exposures assigned to the same grade or risk
pool;
30.4. assess the distribution of obligors or exposures across grades or risk pools and
changes in such distribution as the time goes by.
31.
For the benchmarking purposes the bank shall:
31.1. select appropriate benchmarks and assign internal ratings to them in observance of:
31.1.1. likely differences in the dynamics of the internal rating system the performance of
which is subject to benchmarking and another (benchmark) rating system or in the assignment
process;
31.1.2. likely incompatibility between external ratings’ default frequencies or other realised
risk parameters and respective grades of the internal rating systems or risk pools;
31.1.3. possible differences in used default or loss definitions;
31.1.4. differences between the number of internal rating system’s grades or risk pools,
discriminatory power, calibration accuracy, other characteristics and respective benchmark rating
system’s characteristics;
31.2. use appropriate, regularly updated data covering respective data observation period.
The benchmarking performed by the bank shall cover the period of maximum possible duration;
31.3. document policies explaining validity of applied benchmarking methods, used data
and obtained results. Such documentation shall be updated at least annually.
SECTION IV
VALIDATION OF THE QUANTIFICATION PROCESS
32.
The bank shall carry out regular (at least annual) validation of quantification
process of risk parameters (PD, LGD, CF, EL, etc.).
33.
The bank’s validation process shall cover all stages of the quantification process of
risk parameters: collection of data, estimation of risk parameters, mapping estimated parameters to
grades or risk pools and applying of the mapped parameters to the calculation of capital adequacy.
34.
The bank shall on a regular basis:
34.1. assess the appropriateness of the applicable quantification methods;
34.2. perform back testing with a view to determining the accuracy of calibration;
34.3. carry out benchmarking (except in cases when it is impossible to find benchmarks
suitable for comparison), e.g., the bank can:
34.3.1. compare the reference data set with other data sources;
34.3.2. compare the selected risk drivers with risk drivers selected by others;
34.3.3. compare the values of estimated risk parameters with risk parameters calculated
using other methods (e.g., external models) using the same reference data set;
34.4. assess the stability of risk parameters, i.e. determine the impact of changes in the
reference data set, input parameters, assumptions made or estimation methods as well as other
changes on the values of risk parameters.
35.
For back testing purpose the bank shall:
35.1. compare PD, LGD, CF and other risk parameters established per respective grade
or risk pool with realised default rates, loss severity, CF and other realised risk parameters of
such grade or risk pool;
35.2. determine acceptable intervals for divergence of realised parameters from
respective predicted risk parameters. The bank may apply methods for assessing the accuracy of
calibration specified in Annex 3 and other appropriate methods;
8
9
35.3. identify reasons for the deviation of realised risk parameter values from respective
predicted risk parameters;
35.4. rely on actual data (including data of the bank operating in the Republic of
Lithuania), covering the period of maximum duration;
35.5. take into account the rating system philosophy (i.e. whether the through-the-cycle
or point-in-time ratings are used), on the basis of which the rating system has been developed. The
bank using rating systems based on different rating philosophy shall take into account all
differences between these systems when carrying out the back testing of risk parameters;
35.6. adjust quantified risk parameters, if realised risk parameters remain higher than the
respective predicted values and, where appropriate, apply other means to eliminate shortcomings;
35.7. document the soundness of the applicable back testing methods, used data and
obtained results. Such data shall be updated al least annually.
36.
For benchmarking purpose the bank shall:
36.1. use data which are appropriate to the portfolio, regularly updated and covering the
respective data observation period. The benchmarking carried out by the bank shall cover the
period of maximum possible duration;
36.2. adequately select benchmarks and map risk parameters to them in consideration of
all possible differences between benchmarks and risk parameters;
36.3. document the soundness of applicable benchmarking methods, used data and
obtained results. These documents should be revised at least once a year.
37.
The bank shall assess the observance of requirements set forth in Sections 5–8 and
Subsections 9.2.1–9.2.2, Part II, Chapter IV of the General Regulations for the Calculation of
Capital Adequacy.
SECTION V
VALIDATION OF PROCESSES
38.
The bank shall regularly assess the interaction of the internal rating system with
other processes of the bank and integration of rating systems with other corporate governance
processes. For the purposes of validation of processes the bank shall assess:
38.1. the corporate governance and supervision. The bank shall evaluate the compliance
with the requirements of Section 9.6, Part II, Chapter IV of the General Regulations for the
Calculation of Capital Adequacy and other related requirements;
38.2. the observance of the rating system use requirement in non-capital adequacy
calculation activities (use test), i.e.:
38.2.1. whether rating systems, credit risk parameters used to calculate capital requirement
and related systems and processes are regularly used in non-capital adequacy calculation processes
(in the distribution of capital, in the assessment of risk level the bank intends to assume, bank
strategy, profitability and effectiveness of performance, in the issuance of exposures, making
pricing decisions, management information systems and other processes). If risk parameter values
applied for the internal purposes of the bank differ from values of analogous risk parameters used
for capital adequacy calculation, the bank shall document such differences and reasons thereof;
38.2.2. whether rating systems, credit risk parameters used to calculate capital requirement
and related systems and processes play the key role in non-capital adequacy calculation processes;
38.2.3. how the bank using rating systems in non-capital adequacy calculation activities
observes the experience requirement (experience test), i.e. whether the period of using rating
systems, credit risk parameters used to calculate capital requirement and related systems and
processes for internal purposes of the bank is not shorter than the period specified in Section 1,
Part II, Chapter IV of the General Regulations for the Calculation of Capital Adequacy;
38.3. data management:
38.3.1. assess the appropriateness of the processes of data collection, storage and
aggregation for capital adequacy calculation purposes and the implementation of the IRB approach
requirements established for data, the way in which data consistency, quality, accuracy,
9
10
appropriateness and representativity is ensured. The bank shall assess the observance of
requirements set forth in Subsection 9.1.4, Part II, Chapter IV of the General Regulations for the
Calculation of Capital Adequacy;
38.3.2. evaluate the compatibility of the accounting and risk management systems and
residual differences;
38.3.3. assess the IT infrastructure and other related factors;
38.4. determine the quality of data. The bank shall evaluate the observance of
requirements set forth in Subsection 9.1.4, Part II, Chapter IV of the General Regulations for the
Calculation of Capital Adequacy and other respective requirements regarding the documentation
of rating systems and their elements.
SECTION VI
VALIDATION IN CASE OF SPECIFIC PROTFOLIOS
39.
The bank which for the purpose of calculating the capital required to cover credit
risk of equities applies the internal models approach, shall assess the validity of such internal
models in observance of the following requirements:
39.1. the bank shall have in place sound systems of assessment of accuracy and
consistency of internal models and modelling processes. All material elements of internal models,
modelling processes and their validation shall be properly documented;
39.2. the bank’s internal validation process shall guarantee consistent and meaningful
assessment of the characteristics of internal models and processes;
39.3. quantitative validation methods and data shall be applied in consistent manner. All
changes in calculation and validation methods or used data (as well as of data sources and covered
periods) shall be properly documented;
39.4. the bank shall perform regular back testing comparing actual profitability of
equities (including realised and unrealised profit and loss) with estimated profitability. The bank
shall rely on historical data covering the period of maximum duration. Applicable back testing
methods and data shall be documented, this analysis and documentation shall be updated at least
annually;
39.5. also, the bank shall use other quantitative validation methods and benchmarking.
The benchmarking shall rely upon data that are appropriate to the portfolio, regularly updated and
covering adequate data observation period. The internal assessment of model characteristics
performed by the bank shall be based upon the period of maximum duration;
39.6. the bank shall have in place reliable internal standards, when deviation of realised
profitability from the projected one becomes quite significant. When determining such standards
the bank shall take into account economic cycles and other variations in profitability of equities
that are of systemic nature. All adjustments conforming to internal standards performed after
internal models review shall be documented.
39.7. Liability of parties involved in the modelling process, model approval and review
processes shall be documented.
40.
In case of purchased receivables the bank shall assess the observance of
requirements of Section 9.3, Part II, Chapter IV of the General Regulations for the Calculation of
Capital Adequacy.
41.
When specialised lending exposures are subject to the slotting criteria approach ,
the bank shall assess the observance of requirements of Section 9.5, Part II, Chapter IV of the
General Regulations for the Calculation of Capital Adequacy.
42.
In case of low-default portfolios the bank may:
42.1. use benchmarking to determine the discriminatory power, using other ratings,
results of other models or external information for comparison;
42.2. use expert judgements to determine the accuracy of calibration, in consideration of
internal and (or) external experience in the respective business segment;
42.3. pay more attention to qualitative validation.
10
11
CHAPTER III
VALIDATION OF AMA
43.
The bank intending to apply the AMA to calculate its capital adequacy must meet
the general risk management standards as well as qualitative and quantitative standards applicable
to the AMA, as specified in Part XI of the General Regulations for the Calculation of Capital
Adequacy.
44.
The bank shall have in place the internal AMA validation process enabling to carry
out a detailed assessment whether all elements of the operational risk management system are
functioning properly and reliably.
45.
The operational risk management system validation carried out by the Bank of
Lithuania shall cover the following key elements:
45.1. verifying whether the internal validation process of the bank is carried out in sound
manner;
45.2. satisfying itself that data flows and processes related with the risk management
system are transparent and accessible.
46.
Terms used in this Chapter:
46.1. Operational risk class – means operational risk category which is homogeneous in
terms of the nature of risk and data available for the analysis of such risk (e.g., event type class,
business line class, class of business line events, class of legal units, etc.).
46.2. Operational risk estimate – means distribution of losses identified in each
operational risk class.
46.3. Operational risk measure – means single statistic or parameter extracted from
operational risk estimate. The bank’s overall operational risk capital figure is derived from the
combination of the operational risk estimates calculated for all the operational risk classes.
46.4. Calculation data set – part of the bank’s internal loss events database that is to be
used for the generation of regulatory operational risk estimates and measures.
46.5. Rapidly recovered loss event – means operational risk event that leads to a loss
that is recovered rapidly either partially or completely.
46.6. Multiple time losses – means group of subsequent losses occurring in different
periods of time, but relating to the same operational risk event.
46.7. Multiple-effect losses – means group of associated losses affecting different
entities or units, but relating to the same operational risk event.
46.8. Correlation – means form of interdependence of two or more operational risk
classes (linear and nonlinear, covering all data or only those related with losses in the zone of large
deviations) caused by internal and (or) external factors.
SECTION VII
VALIDATION OF QUALITATIVE STANDARDS
47.
The bank’s operational risk measurement system shall be closely integrated in its
day-to-day risk management process (use test). To this end shall observe the following principles
in its daily activities:
47.1. The AMA method should be applied not only in capital adequacy calculation (e.g.,
the bank can demonstrate how input data, estimates, predictions or outputs generated by the
operational risk measurement system are used in decision-making; and how risk measurement
system is used to manage operational risk in separate business lines);
47.2. the AMA shall be developed in consideration of new experiences gained by the
bank in the application of risk management methods (e.g., the bank can demonstrate that the nature
and extent of the AMA input data adequately reflects specifics of the bank’s business; and that
sensitivity of the risk management system to changes is growing);
11
12
47.3. application of the AMA shall facilitate and enhance the operational risk
management inside the bank (e.g., the bank can demonstrate what decisions have been adopted
with regard to the improvement of processes and control; and that the bank’s internal units have
got familiarised with the operational risk management objectives and activities);
47.4. application of the AMA shall enable to strengthen the bank’s operational risk
control (e.g., the bank can demonstrate that the bank board has taken actions after having obtained
information from the risk measurement system; that the AMA is conducive to the transparency of
activities, risk awareness and operational risk management competence and creates incentives to
improve the operational risk management throughout the bank).
48.
Data used in the operational risk management system may be stored in one or more
databases. The bank’s IT system shall guarantee the possibility of:
48.1.
appropriate availability and maintenance of all relevant databases;
48.2. modelling and estimating the required capacity of databases;
48.3. appropriate control of the data capture process.
49.
With a view to ensuring the possibility, where appropriate, to recover the necessary
information, these IT systems shall be incorporated in the general contingency plans of the bank.
Implemented control procedures should prevent unauthorised data access and ensure the integrity
of data.
50.
Validation of the AMA should involve the verification of observance of the general
requirements for data quality by the bank:
50.1. the bank shall define its own data quality standards and subject them to the
ongoing review and improvement. The bank shall be able to demonstrate that data collected
above minimum thresholds, conforms to the comprehensiveness, appropriateness and accuracy
standards;
50.2. the bank shall perform an independent review of data quality covering control
procedures and systems ensuring the observance of data quality standards;
50.3. the bank shall perform an independent review of the comprehensiveness of internal
data and of the appropriateness of used external data. The bank shall have developed the internal
policies concerning tolerance for any gaps in its internal data.
51.
During validation of the bank’s operational risk management system carried out by
the Bank of Lithuania the bank shall provide with a set of documentation capturing the data
collection and storage policies, descriptions of databases, statement of weaknesses identified
through internal AMA validation and envisaged
follow-up actions for elimination of
shortcomings.
52.
The bank’s AMA should meet the following general quality standards:
52.1. the model shall be applied consistently coordinating it with the operational risk
classes used by the bank;
52.2. the model’s input data should be transparent and easily verified;
52.3. the model should be reliable, i.e. encompass all material factors of operational risk
assumed by the bank and sensitive in responding to significant changes in the nature of the bank’s
operational risk.
SECTION VIII
VALIDATION OF QUANTITATIVE STANDARDS
1. Expected loss
53.
The capital requirement for operational risk will include both expected losses and
unexpected loss. The capital requirement only for unexpected loss may be calculated on condition
that the bank can demonstrate that expected loss is already adequately captured by its internal
practices.
12
13
54.
When determining the compliance of the expected operational risk loss treatment
with the requirement of item 53 above the Bank of Lithuania shall be governed by the following
principles:
54.1. the bank’s expected loss estimates must be consistent with the capital requirement
calculated as the sum of expected and unexpected loss using AMA approved by the Bank of
Lithuania. For operational risk expected loss to be accounted for in any other manner (by means
other than holding capital or establishing provisions) the bank must be able to demonstrate that the
corresponding losses are highly predictable and reasonably stable, and that the estimation process
is consistent over time;
54.2. the maximum offset for operational risk expected loss should be bounded by the
expected loss exposure calculated by the bank’s AMA approved by the Bank of Lithuania;
54.3. allowable offsets for expected loss must be clear capital substitutes or otherwise
available to cover expected loss with a high degree of certainty over a one-year time horizon.
Where the offset is something other than provisions, its availability should be limited to those
operations with highly predictable, routine losses. Because exceptional operational risk losses do
not fall within expected loss, specific reserves for any such events that have already occurred will
not qualify as allowable offsets;
54.4. the bank is expected to clearly document how its operational risk expected loss is
measured and accounted for, including how any expected loss offsets meet the conditions outlined
above.
2. Confidence level
55.
The operational risk measurement must capture potentially severe tail events
achieving the soundness standard comparable to 99.9% confidence level in one-year period.
56.
The bank may perform direct calculations corresponding to 99.9% confidence level
in one-year period. Where this is impracticable, the bank may calculate the initial operational risk
measure at a lower confidence level on the right side of the distribution of loses covering low
probability high severity events and then scale it up to the 99.9 % using appropriate methods.
57.
. The confidence level at which the initial operational risk measure is computed
should be located in the right-end of the distribution of the losses.
58.
If scaling is used, the bank should be able to demonstrate that scaling technique is
applied in transparent and reliable manner and that the model outputs are valid and correct.
3. Four main elements of AMA
59.
The bank’s operational risk measurement system must include all of the four main
elements: internal data, external data, scenario analysis and business environment and internal
controls factors. The bank shall decide upon the specific weight of each of the aforementioned
elements in its operational risk measurement system and how they are combined with each other.
60.
The model must be documented in detail, and documentation regularly updated.
These documents must define how the main four elements are combined with each other and the
specific weight assigned to them, as well as the description of the process modelling illustrating
the use of the four elements.
61.
The bank may use qualitative data as one component part of the four main elements
(e.g., in situations which are assessed and described by means of qualitative characteristics). In
such case the bank must be able to demonstrate that:
61.1. it has experts competent to assess the qualitative data;
61.2. it has taken measures to eliminate deviations;
61.3. qualitative data are appropriate for clearly defined risk variables;
61.4. qualitative data conform to planned risk management objectives (e.g., if for the
purpose of recording internal fraud scoring cards are used, the consistency of the assigned
estimate to incurred losses should be monitored over time).
13
14
3.1. Internal data
62.
Internally generated operational risk measures shall be based on a minimum
historical observation period of five years. When a bank first moves to an Advanced Measurement
Approach, a three-year historical observation period is acceptable. Because the amount of data
collected in the low-frequency operational risk class may be insufficient, this class may be subject
to longer minimum historical data observation period. In case of data shortages the bank must use
conservative operational risk measures.
63.
The bank must have in place the policies for the integration of losses recorded in
the internal loss event database into the calculation data set. The Bank of Lithuania shall be
furnished with the respective information about the banks policy for identification and
classification of losses.
64.
The bank should be able to distinguish those operational risk events that are related
with the use of insurance and other risk transfer mechanisms. The Bank of Lithuania may allow
the bank to exclude data about rapidly recovered loss events from the calculation data set.
65.
Multiple time losses before integrating them in the calculation data set should be
aggregated into a single loss. Multiple-effect losses before integrating them in the calculation data
set should also be aggregated into a single loss. Possible exceptions provided by the bank must be
documented.
66.
The bank must have set specific standards for assigning loss data related with
centralised functions or activities capturing more than one business line. This can be done in
several ways, e.g., mapping all losses to that business line on which they have the most severe
impact, or distributing losses proportionately among the affected business lines.
67.
The bank must define appropriate minimum loss thresholds for internal loss data
collection. The level of these thresholds shall depend upon complexity of typical and operational
risk class, and additionally consideration may be given to cost-benefits analysis in the collection of
data below the minimum threshold. For validation purposes the soundness of definition of the
minimum thresholds shall be assessed and it shall be analysed whether such thresholds do not have
material impact on model outputs. The bank must demonstrate that:
67.1. the minimum thresholds are acceptable and adequately reflect the type of risk;
67.2. the model captures all material operational risk event losses;
67.3. selected minimum thresholds do not negatively affect the accuracy of the
operational risk measures.
68.
The Bank of Lithuania shall assess whether the bank avoids possible biases in the
estimation of the model parameters taking into account the incompleteness of the calculation data
set predetermined by the applicable minimum thresholds.
3.2. External data
69.
The bank’s operational risk measurement system shall use relevant external data.
Such data may be derived from public sources of information, entities providing specialised
information services or other credit institutions, e.g., consortium data.
70.
The bank using external data must satisfy itself that they are classified according to
the same principles as the internal data of the bank and that information is comprehensive and
reliable. External data shall be used when the bank’s internal data are insufficient (e.g., having
started new business). Upon incorporating external data in the risk measurement system the bank
shall assess the differences in the scope of its own activities and activities of the credit institution
supplying the data and respectively adjust the data.
71.
When bank data on low probability high severity losses and in particular on their
causes held in the single information exchange system are insufficient, useful additional
information might be obtained from public sources of information. The bank using the data public
14
15
sources of information must satisfy itself that they are appropriate, unbiased and relevant to the
bank’s risk profile.
3.3. Scenario analysis
72.
Scenario analysis in conjunction with external data shall be used in the first instance
to evaluate the bank’s exposure to high severity events; nevertheless it may also be used as one of
the information sources in determining the overall operational risk exposure of the bank.
73.
The bank‘s scenario analysis should be aimed at reducing to the minimum the
effects of subjectivity and biases. Scenario assumptions must be based on empirical evidence.
Scenario building may capture relevant internal and external data. Scenario assumptions and
process must be properly documented.
3.4. Business environment and internal control factors
74.
Business environment and internal control (BEIC) factors used in the operational
risk system must reflect potential operational risk growth sources, such as rapidly increasing
business volume of the bank, introduction of new products, staff turnover, and system downtime.
75.
BEIC factors should reflect the reduction (or increase) in assumed operational risks
due to the impact of internal and external factors. BEIC factors may be incorporated in the
operational risk measurement system in different ways, e.g., trough key risk indicators. The bank
shall properly document the place of BEIC factors in its operational risk measurement system.
76.
The operational risk measurement system must capture at least those BEIC factors,
which have significant influence on the type of operational risks.
4. Correlation
77.
Correlations of individual operational risk estimates shall be recognised only when
the bank is able to demonstrate to the satisfaction of the Bank of Lithuania, that correlation
measurement systems used by it are reliable, implemented maintaining integrity and take into
account the uncertainty surrounding such correlation estimates, particularly in periods of stress.
78.
The bank which has no correlation assumptions in its AMA, shall calculate the
general capital requirement for operational risk by summing up individual operational risk
estimates.
79.
The bank which makes correlation assumptions in its AMA, shall take into account
the following conditions:
79.1. model documentation shall define and justify correlation assumptions and assess
model sensitivity to these assumptions;
79.2. the interdependence of low probability high severity events (tail events) should be
analysed with particular care applying appropriate quantitative and qualitative methods which
may differ from those applied in measuring correlation of body events, because these events have
different nature.
5. Insurance and other risk transfer mechanisms
80.
The outsourced activities shall not be considered part of other risk transfer
mechanisms.
81.
The bank shall monitor on a regular basis the use of insurance and other risk
transfer mechanisms and recalculate the capital requirement for operational risk, when the
conditions of use and scope of application of the aforementioned mechanisms considerably
changes.
SECTION IX
15
16
INTERNAL VALIDATION OF THE BANK’S OPERATIONAL RISK
FRAMEWORK
82.
The bank shall be responsible for the organisation of the internal AMA validation
(hereinafter – internal validation) process, which should be carried out in observance of clear
methodology established by the bank (including frequency of internal validation). This
methodology shall be properly documented.
83.
The bank shall carry out the internal validation on the basis of the following
principles:
83.1. internal validation methods selected by the bank shall correspond to the extent of
the bank's risk and suitable for application in the developing business environment;
83.2. the internal validation shall incorporate both, quantitative and qualitative elements;
83.3. the internal validation processes and results shall be subject to independent review.
84.
The frequency of internal validation shall depend upon the relevance of the
validation element within the bank’s operational risk management framework.
85.
The bank shall regularly review and update its internal validation methodology.
Certain parts of risk measurement system and risk management processes must be revalidated at
least in the event of material changes in the nature of the bank’s operational risk and (or) model
methodology, assumptions made or management process.
86.
For the purpose of internal validation the bank shall satisfy itself that all data used
in the operational risk measurement framework (including actual and modelling data, results of
scenario analysis, BEIC factors) which exceed the established minimum thresholds, are consistent,
appropriate and accurate, that assumptions are free from material bias and results are realistic.
87.
Model validation shall ensure that relationship between model inputs and outputs is
logically justified and stable and model methods - transparent.
88.
The bank’s internal validation should also incorporate the assessment of adequacy
of the bank’s operational risk management processes. The bank shall demonstrate to the Bank of
Lithuania that:
88.1. risk management documentation is complete;
88.2. management information reporting procedures are followed;
88.3. captured loss data conform to required data standards;
88.4. the bank makes relevant adjustments in consideration of outcomes generated by the
risk measurement system;
88.5. the bank revises and updates the operational risk management procedures in timely
manner;
88.6. the key risk indicators, loss data and risk estimates are consistent with the results of
internal validation performed by the bank.
CHAPTER IV
FINAL PROVISIONS
89.
The first validation of the IRB approach shall be carried out by the bank before
applying the IRB approach for capital adequacy calculation. When applying with the Bank of
Lithuania for granting the permission to apply the IRB approach, the bank shall furnish the report
on performed validation of the IRB approach. Having developed new credit risk assessment
systems or systems not yet validated by the bank before applying for the permission to apply the
IRB approach, the bank may provide validation methods used in the process of development of the
credit risk measurement system and results of such methods and (or) plans specifying when and
what validation methods to be applied in future.
90.
Detailed internal validation of the elements of the bank’s operational risk
management system shall be carried out before applying for the permission to apply the AMA
approach to the calculation of capital adequacy. When filing an application for granting the
16
17
permission to apply the AMA, the bank shall submit the performed internal AMA validation
report.
17
18
Annex No. 1
to the Regulations on Validation and Its
Assessment
IRB APPROACH VALIDATION ELEMENTS
Assessment of
validation process
carried out by the
Bank of Lithuania
IRB approach
validation process
carried out by the
bank
Validation of
rating systems
Rating system
structure and
assignment process
Validation of
processes
Quantification of
risk parameters
Corporate
governance
and
supervision
priežiūra
Practical use
of rating
systems
Data
maintenance
Areas of use
Data
gathering
Scope of use
Accounting
systems
Experience
requirement
IT
infrastructure
Quality of
documents
Data collection
Structure
Assignment
methods
Estimation of risk
parameters
Mapping of risk
parameters to
grades or risk
pools
Use for capital
adequacy
calculation
18
19
Annex No. 2
to the Regulations on Validation and Its
Assessment
DISCRIMINATORY POWER ASSESSMENT METHODS
1. Cumulative accuracy profile curve (hereinafter – CAP curve) – validation method
used to assess discriminatory power which shows the relationship between cumulative percentage
share of the defaulted obligors and cumulative percentage share of all obligors. Using CAP curve
the bank can determine the accuracy ratio, i.e. the ratio of the area between the diagonal and rating
system CAP curve and the area between the diagonal and ideal rating system CAP curve. The
interval of accuracy ratio values is [0;1], higher accuracy ratio is the indicator of stronger
discriminatory power.
2. Receiver operating characteristic curve (hereinafter – ROC curve) – validation
method used to assess discriminatory power which shows the relationship between cumulative
percentage share of correctly predicted defaults in the general composition of actual defaults and
cumulative percentage share of wrongly forecasted defaults in the general composition of actual
non-defaults. Using ROC curve the bank can assess the accuracy ratio within [0;1] interval, higher
accuracy ratio is the indicator of stronger discriminatory power.
3. Curve of concordance (hereinafter – COC curve) – validation method used to assess
discriminatory power which shows the relationship between cumulative percentage share of the
defaulted obligors and cumulative percentage share of non-defaulted obligors. Using COC curve
the bank can assess the coefficient of concordance, obtained as the ratio of the area between the
diagonal and COC curve of the rating system to the area between the diagonal and ideal rating
system COC curve. The interval of the values of the concordance coefficient is [0;1], higher
coefficient of concordance is the indicator of stronger discriminatory power.
4. Information entropy based methods. Entropy implies the extent of uncertainty that is
reduced applying the rating system or model subject to validation. For the purpose of applying the
entropy-based methods the bank may use conditional entropy, conditional information entropy
ratio, mutual information entropy ratio and information value.
5. Brier score method – validation method used to assess discriminatory power, when the
mean squared difference of the forecasted PD for obligor grade or risk pool and non-default or
default indicator value (i.e. 1 in case of default, and 0 in case of non-default) is estimated. The
lower Brier score value is the indicator of stronger discriminatory power, with the values varying
within the range of [0;2].
6. Alpha/beta (classification) error rate – validation method used to assess
discriminatory power with the help of which the minimum probability of error is determined
relying on assumption that rating system results can be only of two types (i.e. non-default or
default). This probability is equal to the sum of percentage share of defaulted obligors , recognised
on the rating system basis as having potential to be non-defaulting, in the general composition of
defaults (alpha) multiplied by respective probability and percentage share of non-defaulted
obligors , recognised on the rating system basis as having potential to be defaulting, in the general
composition of non-defaults (beta) multiplied by respective probability. Alfa/beta (classification)
error values vary within the interval of [0;1], lower values are the indicator of stronger
discriminatory power.
Annex No. 2
to the Regulations on Validation and Its
Assessment
19
20
METHODS FOR ASSESSMENT OF ACCURACY OF CALIBRATION
1. Binomial test – is a statistical backtesting method used to determine whether changing
values of risk parameters per respective grade or risk pool fall within a certain interval. This test
can be applied to one rating scale grade or risk pool and shall be based on assumption that defaults
are independent events. If, say there are 1,000 obligors in a respective grade or risk pool and PD of
each obligors is 1%. Then the estimated number of defaults will be ten. The actual number of
defaults would be different depending upon the number of obligors . The bank using the binomial
test can determine the number defaults per respective grade or risk pool, when there’s certain
statistical significance level and determined confidence interval.
2. Normal test – is a statistical backtesting method used to determine whether mean of
quantified values of a respective risk parameter is reliable in consideration of standard deviation of
actual risk parameters. Assuming there are 1,000 obligors in a respective grade or risk pool and PD
of each borrower is 1%. Realised default rates of the past 5 years were 1.2, 1.8, 0.8, 0.7 and 1.1%
respectively. The normal test helps to assess the reliability of average PD in consideration of
historical standard deviation of default rates.
3. Hosmer–Lemeshow test – is a statistical backtesting method used to determine whether
varying values of risk parameters of respective grades or risk pools fall within a certain interval.
This test may be applied to more than one rating scale grade or risk pool based on the assumption
of independence of default events.
4. Traffic light approach – is a statistical backtesting method whereby the distribution of
one year default rates is approximated to normal distribution and in accordance with confidence
interval the certain number of defaults is mapped to respective traffic light colours. On the basis of
distribution of defaults of respective colours the accuracy of forecasted PD estimate can be
determined.
20