IAEA Training Course on Safety Assessment of NPPs to Assist Decision Making
Overview of Deterministic Safety Analysis:
Sensitivity & Uncertainty Analysis, Q.A.
(Part. 3)
Lecturer
Lesson IV 2_3
Workshop Information
IAEA Workshop
City , Country
XX - XX Month, Year
Sensitivity & Uncertainty Analysis
– A best-estimate analysis must include an uncertainty analysis
(UA).
– Results calculated in the analysis are given an “uncertainty
interval”, i.e. one in which it is expected with high
probability to find the “true” result.
– Uncertainties in the input data of the analysis and in the
calculational devices (codes) are propagated to the calculated
results.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
2
Sensitivity & Uncertainty Analysis
– Sensitivity analysis (SA): “what-if” analysis. Impact
of changes of some attribute (input data, models,
numerics, etc) of an analysis or calculation on the
results.
– Relation among SA and UA.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
3
Sensitivity & Uncertainty Analysis
– Basic ingredients of an UA:
• Establishing the input uncertainties:
 From input parameters
 Associated with the calculational device(s): from models and
calculational patterns.
• Combine input uncertainties to find the results uncertainty.
– When the number of uncertain attributes is very
high

selection
of
those
most
influential
(on important results). One tool  Sensitivity analysis.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
4
Sources of Uncertainty in a Safety Analysis
– U on initial (i.e. previous to the accident) conditions U of the
plant, and on boundary conditions in the transient.
– U on physical parameters describing the plant, its systems and
materials: geometry, state and transport properties, etc.
– U coming from the inaccuracy of physical models and
correlations, including errors due to scaling and extrapolation.
– Errors coming from numerics of the codes.
– Errors due to omissions in the models or the noding.
– U due to the computer and the user.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
5
Sources of Uncertainty in a Safety Analysis
– De-compensation of errors, arising from scaling or
extrapolation.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
6
CSAU Methodology
– USNRC sponsored the development of a methodology for
uncertainty quantification in complex codes calculations:
Code Scaling, Applicability and Uncertainty Evaluation
Methodology (CSAU).
– Formerly developed for BE LBLOCA analyses, but can be
applied to other scenarios. It is a study of:
• Capability of the code to scale up phenomena from smallscale facilities to nuclear plants.
• Applicability of the code to a particular scenario + plant.
• Uncertainty on the important calculated results (e.g. PCT).
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
7
CSAU Methodology
Systematic approach with three succesive phases:
1. REQUIREMENTS AND CAPABILITIES
• Scenario specification: temporal windows and most important
phenomena.
• Plant selection.
• Phenomena and processes identification and ranking (by impact on
the safety criteria for the scenario). PIRT is established: it will guide
the uncertainty quantification. PHENOMENA and processes found
by examining exp data and code simulations. RANKING through
experts opinion, decision-making methods (AHP) and calculations.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
8
CSAU Methodology (Req. and capabilities)
• Frozen code version selection: changes allowed for correction only.
• Provision of complete code documentation, including MC/QE: Models
and Correlations Quality Evaluation.
• Code’s capabilities are established:
 Field equations : can model global processes (such as multi-D flows,
boron injection, etc) ?.
 Closure equations: do they enable the code to calculate important
processes ?.
 Numerics.
 Structure and nodalization: do they allow the modelling of important
design characteristics ?.
• Determination of code applicability, by comparing code’s capabilities
to scenario requirements
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
9
CSAU Methodology (Req. and capabilities)
2. ASSESSMENT AND RANGING OF PARAMETERS
– Establishment of Assessment Matrix, that must include both
separate and integral effects tests. Necessary for evaluating:
• Code accuracy to calculate important phenomena (isolated or
interacting). PIRT is used.
• Code capability to scale-up the phenomena to plant conditions.
Counterpart tests are quite useful.
• Influence of nodalization.
– Plant nodalization definition: compromise between economy
and capture of the phenomenology. An iterative process is
employed. Where the data base is insufficient to judge
nodalization, a separate bias may be added in determining
uncertainty.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
10
CSAU Methodology
(Assess. & Rang of parameters)
– Definition of code and experimental accuracy:
• Individual uncertainty contributions (arising from code and
experiment) are estimated from experimental data. The
actual probability distribution may not be easily quantified.
• The individual uncertainty of each contributor is input to the
plant model, and the effect upon important safety criteria
evaluated by separate plant calculations.
• SET and IET simulations can be a confirmatory support of
these estimations.
– Determination of effect of scale, quantifying it for bias and
deviation as a contributor to overall uncertainty.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
11
CSAU Methodology
(Assess. & Rang of parameters)
– Determination of effect of scale, quantifying it for bias and
deviation as a contributor to overall uncertainty.
• Lack of data  bounding analysis to provide a separate uncer.
Bias
• MC/QE document + code assessment reports  identification
of closure relations and evaluation of the capability to scaleup phenomena from PIRT.
• Test facility scale distortions do not have the same effect
throughout a transient. Results not affected by scale are used
directly to evaluate the total uncertainty; those affected are
subject to further examination.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
12
CSAU Methodology
3. SENSITIVITY AND UNCERTAINTY ANALYSIS
– Determination of the effect of reactor input parameters and state
• Uncertainties come from plant operating state at the initiation of the
transient (e.g. State of fuel is a function of the burnup history and of
the original manufacture tolerances) and also from process variables
• Uncertainties quantified through experimental data and/or analytical
studies. Biases and distributions, or sparate (bounding) biases.
– Performance of plant sensitivity calculations: sensitivity of
primary safety criteria parameters to various plant operating
conditions that arise from uncertain initial state; also to scale and
process variables.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
13
CSAU Methodology (SA & UA)
– Determination of combined bias and uncertainty:
• The individual uncertainties (coming from code limitations, scale
effects, input variations, etc) are combined.
• Proven technique: pdf determination of a safety parameter through
Monte Carlo sampling of a response surface that “substitutes” the
code.
• Economic considerantions can recommend take an individual
uncertainty as a separate bias based on bounding sensitivity
calculations.
– Determination of total uncertainty: an error band, or a statement
of probability for the limiting value, is given for each safety
criteria parameter.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
14
CSAU Methodology (SA & UA)
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
15
CSAU Methodology (SA & UA)
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
16
CSAU Methodology (SA & UA)
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
17
Uncertainty Analysis - UMS
–
Uncertainty Methods Study (UMS): supported by Committee on
the Safety of Nuclear Installations (CSNI) from OECD Nuclear
Energy Agency. Aimed to compare several uncertainty analysis
methodologies, from:
•
•
•
•
AEA Technology (UK)
GRS (Germany)
CEA (France)
ENUSA (Spain)
• Pisa University (Italy)
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
18
Uncertainty Analysis - UMS
–
Five methods:
• Pisa method: extrapolation from integral experiments.
• GRS, IPSN, ENUSA methods: identify and combine
input uncertainties, using subjective pdfs.
• AEAT method performs a bounding analysis.
–
The five methods have been applied to the simulation
of a cold leg SBLOCA experiment in the ROSA-IV
Large Scale Test Facility (LSTF).
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
19
Uncertainty Analysis - UMS
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
20
Uncertainty due to User Effect
– Any differences in calculations that use the same code
version and the same specifications (e.g. Initial and boundary
conditions) for a given plant or facility.
– Code users have a significant influence on calculation
results.
– User effects have been identified in numerous publications as
the origin of many calculation failures.
– In the many Standard Problems proposed by CSNI appear
the influence of the code user.
– The new generation of the advanced computer codes have
reduced, but by no means suppress the user effect.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
21
Uncertainty Due to User Effect
–
Reasons:
• Code user guidelines not fully detailed or comprehensive
• Experienced users may overcome code limitations adding
“engineering knowledge” to the input deck.
• System nodalization: many times, the user must model 3D
geometries using 1D components.
• Application of steady state qualified models to transient
conditions.
• Lack of understanding of the code capabilities and limitations.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
22
Uncertainty due to User Effect
–
Reasons:
•
Code options and physical model parameters: there are several
options (model and correlations) for the user to choose.
Uncertain parameters (e.g. Pressure loss coefficients) must
also be specified by the user.
•
Effect of compiler and computer
•
Lack of information about facilities and experiments
•
Error bands and the values of initial and boundary conditions
badly defined
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
23
Uncertainty due to User Effect
–
Reasons:
•
Input parameters for system characteristics: e.g.
Distribution of heat losses.
•
Input parameters for system components: a number of
empirical models (pumps, valves,etc) are specified by the
user, sometimes based on extrapolation from scale devices.
•
Specification of initial and boundary conditions:
sometimes users fail to obtain a steady state previous to the
transient.
•
Specification of state and transport properties.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
24
Uncertainty Due to User Effect
–
Reasons:
• Time step size choice: if a code is using an explicit
numerical scheme, the results may vary significantly with the
time step size.
• QA guidelines: should be followed to check the correctness
of the values in the input decks, despite the automatic
consistency checks provided by the code.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
25
Quality Assurance in Development of
Evaluation Models
EVALUATION MODEL DEVELOPMENT AND
ASSESSMENT PROCESS (EMDAP)
–
Draft Regulatory Guide DG-1096, released Dec. 2000, gives
recommendations for the process of development and assessment
of Evaluation Models (transient and accident analysis methods).
–
One of the basic principles during the EMDAP is following an
appropriate Quality Assurance (QA) protocol.
–
QA standards are a key feature of the development and
assessment process.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
26
Quality Assurance in Development of
Evaluation Models
–
QA Plan: covers the procedures for design control,
document control, software configuration control and
testing and error identification and corrective actions
used in the development and maintenance of the EM.
–
The QA Plan also ensures adequate training of
personnel involved with code development and
maintenance as well as those who perform the
analyses.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
27
Quality Assurance in Development of
Evaluation Models
–
–
An appropriate QA protocol must be established early in
the development and assessment process.
Development, assessment, maintenance and application of
an EM are activities related to the requirements of the
Appendix B (“Quality Assurance Criteria for Nuclear
Power Plants and Fuel Reprocessing Plants”) to the US
code of federal regulations 10 CFR 50:
• Section III (”Design Control”) of Appendix B is a key
requirement for this activity, stating that design control
measures must be applied to reactor physics, thermal,
hydraulic and accident analyses.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
28
Quality Assurance in Development of
Evaluation Models
– Section III states: “The design control measures shall provide
for verifying or checking the adequacy of design, such as by the
performence of design reviews, by the use of alternate or
simplified calculational methods, or by the performance of a
suitable testing program”.
– Section III also states that design changes should be subject to
appropriate design control measures.
– Section V: requires documented instructions, e.g., user guides
– Section VI: address Document Control
– Section XVI (“Corrective Action”): errors must be promptly
identified and corrected, and corrective actions must be taken to
preclude repetition. All this must be documented.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
29
Quality Assurance in Development of
Evaluation Models
Section XVII: address QA Records
–
In order to fulfill the App. B requirements, independent peer
reviews should be performed at key steps in the process.
This is an important point when complex codes are involved.
–
PEER REVIEW: an evaluation technique in which software
requirements, design, codes or other products are examined
by persons whose rank, responsibility, experience and skill
are comparable to that of the authors.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
30
Quality Assurance in Development of
Evaluation Models
–
Composition of a peer review team: programmers,
developers and users, and independent members with
recognized expertise in relevant engineering and science
disciplines, code numerics and computer programming.
–
Experts in the Peer Review team not directly involved in the
EMDAP can enhance the robustness of the EM and can be
valuable in identifying deficiencies that are common to
large system analysis codes.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
31
Quality Assurance in Development of
Evaluation Models
–
In the early stages of EM development it is recommended
that a review team be convened to review EM requirements,
which are the basis of the EMDAP. Such requirements are:
• Analysis Purpose, Transient Class and Power Plant Class.
• Figures of merit (i.e. Quantitative standards of acceptance)
used to define acceptable answers for a safety analysis.
• Systems, components, phases, geometries, fields and
processes that must be modeled.
• Ranking of key phenomena and processes.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
32
Quality Assurance in Development of
Evaluation Models
–
Configuration control
development process:
practices
throughout
the
• Adopted to protect program integrity and allow
traceability of both the code version and the plant input
deck.
• Responsibility for these functions should be clearly
established.
• At the end of the process, only the approved, identified
code version and plant input deck should be used for
licensing calculations.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
33
Quality Assurance in Development of
Evaluation Models
–
The QA Plan documentation must include process that
address all of the topics described.
–
A Safety Analysis must be adequately documented:
• All sources of data referenced and documented.
• Whole process recorded and archived to allow independent
checking.
• Results of the SA structured and presented in an appropriate
format to provide a good understanding of the course of the
events, and to allow easy checking of the individual
acceptance criteria.
IAEA Training Course on Safety Assessment of NPPs to assist Decision Making
34