The Risk Driver Approach to Project
Schedule Risk Analysis
A Webinar presented by
David T. Hulett, Ph.D.
Hulett & Associates, LLC
To the
College of Performance Management
April 18, 2013
© 2013 Hulett & Associates, LLC
1
Risk Drivers Method Agenda







Limitations of traditional 3-point estimating
Explain “Risk Drivers” approach, start with the risks themselves
– One Driver per activity
– Multiple Risk Drivers per Activity
Correlation is modeled in the Monte Carlos simulation
Assess the schedule against CPM scheduling best practice
Simple space vehicle development schedule as an example
Prioritize risks to schedule
Mitigation of high-priority risks
© 2013 Hulett & Associates, LLC
2
Limitations with the Traditional
3-point Estimate of Activity Duration


Typical schedule risk analysis starts with the activity that is impacted
by risks, not the risks themselves
– Estimates the 3-point estimate for optimistic, most likely and
pessimistic duration
– Creates a probability distribution for activity duration
– Performs Monte Carlo simulation
Which risks cause the most overall schedule risk? These questions
are typically answered by:
– Sensitivity to activity durations
– Criticality of activity durations
© 2013 Hulett & Associates, LLC
3
Some Problems with Traditional Approach



Can tell which activities are crucial, but not directly which
risks are driving
Makes poor use of the Risk Register that is usually available
Cannot decompose the overall schedule risk into its
components BY RISK
– Ability to assign the risk to its specific risk drivers helps with
communication of risk causes and risk mitigation
© 2013 Hulett & Associates, LLC
4
We Propose a Different Approach: Start with
the Risks Themselves

Drive the schedule risk by the risks already analyzed in the Risk
Register
For each risk, specify:

Starting with the risks themselves gives us benefits

– Probability it will occur
– Impact on time if it does
– Activities it will affect
– Links qualitative analysis to the quantitative analysis
– Estimates the impact of specific risks for prioritized mitigation purposes
– Correlations between activities happen automatically – never have to
guess at these coefficients again, never get impossible matrices
© 2013 Hulett & Associates, LLC
5
Simple Example of Risk Register Risks




Use the Risk Drivers feature in Pertmaster 8
Collect probability and impact data on risks
Load the risks
Assign risks to schedule activities
© 2013 Hulett & Associates, LLC
6
Risk Drivers Mechanics (1)

The risk Driver is assigned to one or several activities,
affecting their durations by a multiplicative Driver
– E.g., the Driver may be .90 for optimistic, 1.0 for most likely and
–


1.25 for pessimistic
These Drivers multiply the schedule durations of the activities to
which they are assigned
Risks can be assigned to one or more activities
Activity durations can be influenced by one or more risks
© 2013 Hulett & Associates, LLC
7
Risk Drivers Mechanics (2)

Risk Drivers are assigned a probability of occurring on any
iteration.
– When the risk occurs, the Driver used is chosen at random from
–

the 3-point estimate and operates on all activities to which it is
assigned
When not occurring on an iteration the risk Driver takes the value
1.0, a neutral value
When an activity is influenced by more than one risk, their
Drivers are multiplied together, if they happen on an
iteration
© 2013 Hulett & Associates, LLC
8
Risk Driver
Probability is 100%, Driver can be + or Entire Plan : Duration
100% 115
90% 109
85% 108
350
80% 106
75% 105
70% 104
300
60% 103
250
55% 102
50% 101
200
45% 101
40% 100
Cumulative Frequency
65% 104
Hits
Here the
Ranges are
based on
deviations +
and – from the
Plan.
Probability is
100%
95% 111
400
For the examples
we use an activity
with 100 days in
the schedule
35% 99
150
30% 99
25% 98
100
20% 97
15% 96
50
10% 95
5% 94
0
0% 90
90
95
100
105
110
115
Distribution (start of interval)
© 2013 Hulett & Associates, LLC
9
Risk Driver
Prob. = 100%, Driver is all Overrun
Entire Plan : Duration
100% 130
350
95% 125
90% 122
80% 119
75% 118
70% 117
250
60% 115
200
55% 114
50% 113
45% 112
150
40% 111
Cumulative Frequency
65% 116
Hits
Here the
Plan is the
Optimistic
Value.
Probability is
100%
85% 121
300
35% 110
30% 109
100
25% 109
20% 108
15% 107
50
10% 105
5% 104
0
100
0% 100
105
110
115
120
,
Distribution (start of interval)
125
130
10
Assigning a Probability Less than 100%

The essence of a “risk” is its uncertainty in two dimensions:
– Uncertainty of its occurrence, specified by a probability
– Uncertainty of its impact, specified by a range of durations

If the risk may or may not occur, we specify the probability
that it will occur
– The risk occurs and affects the activities it is assigned to on X% of
–
the iterations, chosen at random, the multiplicative Driver used is
chosen at random from the range of data input by the user
On (1 – X)% of the iterations, Driver takes 1.0 value
© 2013 Hulett & Associates, LLC
11
Assigning a Probability Less than 100%
Entire Plan : Duration
Spike
contains
40% of
the
probability
Entire Plan : Duration
2000
100% 130
100% 114
95% 123
95% 107
90% 120
90% 104
3500
85% 118
85% 101
80% 116
1600
80% 100
3000
75% 114
75% 100
70% 113
1400
70% 100
1200
Hits
55% 109
50% 107
1000
45% 106
40% 103
800
65% 100
2500
60% 100
55% 100
Hits
60% 110
Cumulative Frequency
65% 111
2000
50% 100
45% 100
40% 100
1500
35% 100
Cumulative Frequency
1800
Spike
contains
70% of
the
probability
35% 100
30% 100
30% 100
600
25% 100
1000
25% 100
20% 100
400
20% 100
15% 100
15% 100
500
10% 100
200
10% 99
5% 100
0
100
0% 100
105
110
115
120
Distribution (start of interval)
125
130
5% 97
0
0% 90
90
95
100
105
110
115
Distribution (start of interval)
© 2013 Hulett & Associates, LLC
12
Assigning
More than One Risk to an Activity



If more than one risk is acting on an activity, the resulting
ranges are the multiplication of the percentages
This is reality – an activity is often affected by multiple risks
Two cases are shown next:
– When both risks are 100% likely to occur
– When both risks are < 100% likely to occur

In each case, the computer simulation creates the
uncertainty range on an activity’s duration – it is not
estimated
© 2013 Hulett & Associates, LLC
13
Parallel and Series Risks – Multiplicative –
Used with Risk Drivers (RiskDrivers)
Risk 1 1.2 Driver
Risk 2 1.05 Driver
Use 1.2 Driver, the largest
Driver only
If these two risks are parallel, they can be recovered
simultaneously
Risk 1 1.2 Driver
Risk 2 1.05 Driver
Use (1.2 x 1.05 = 1.26)
Driver, multiply the two
If these two risks are series, they can not be
recovered simultaneously
© 2013 Hulett & Associates, LLC
14
Two Risks affect One Activity using Drivers that
Occur 100% - placed in Series
Entire Plan : Duration
100% 146
260
95% 131
90% 127
240
85% 124
220
80% 123
75% 121
200
70% 119
180
60% 117
160
55% 116
140
50% 115
45% 114
120
40% 112
100
Cumulative Frequency
65% 118
Hits
Risks in
series, P80
is 123 days
35% 111
30% 110
80
25% 109
60
20% 108
40
15% 106
10% 104
20
5% 102
0
0% 92
100
110
120
,
Distribution (start of interval)
130
140
15
Two Risks affect One Activity using Drivers
that Occur 100% - in Parallel
Entire Plan : Duration
100% 130
95% 125
350
90% 122
85% 121
80% 119
300
75% 118
70% 117
60% 115
Risks in
parallel, P80
is 119 days
Hits
55% 114
200
50% 113
45% 112
40% 111
150
Cumulative Frequency
65% 116
250
35% 111
30% 110
100
25% 109
20% 108
15% 107
50
10% 106
5% 105
0
100
0% 100
105
110
115
120
125
130
Distribution (start of interval)
© 2013 Hulett & Associates, LLC
16
Two Risks with Less than 100% Probability
Affecting one Activity – Risks in Series
Entire Plan : Duration
100% 144
1800
90% 119
1600
85% 116
80% 114
1400
75% 112
70% 110
1200
60% 107
1000
55% 105
50% 103
45% 101
800
40% 100
Cumulative Frequency
65% 108
Hits
The spike at 100
days represents (1)
the likelihood that
neither risk occurs
[60% x 50% =
30%] and (2) the
chance that 100
days is picked
when one or both
occur.
95% 123
35% 100
600
30% 100
25% 100
400
20% 100
15% 100
200
10% 99
5% 96
0
0% 90
90
100
110
120
130
140
Distribution (start of interval)
© 2013 Hulett & Associates, LLC
17
Two Risks with Less than 100% Probability
Affecting one Activity – Risks in Parallel
Entire Plan : Duration
100% 130
95% 122
2200
90% 118
85% 115
2000
80% 113
1800
75% 111
70% 110
1600
60% 107
1400
55% 105
1200
50% 103
45% 101
1000
40% 100
Cumulative Frequency
65% 109
Hits
With one parallel
risk’s having a
minimum range of
100%, it cannot be
less than 100 days
35% 100
800
30% 100
600
25% 100
20% 100
400
15% 100
10% 100
200
5% 100
0
100
0% 100
105
110
115
120
125
130
Distribution (start of interval)
© 2013 Hulett & Associates, LLC
18
Risk Drivers Model How Correlation Occurs
Coefficients are Calculated (1)
Risk Probability = .5,
Range .95, 1.05, 1.15
Activity 1
Activity 2
Correlation = 100%
© 2013 Hulett & Associates, LLC
19
Scatter showing 100% Correlation
© 2013 Hulett & Associates, LLC
20
Risk Drivers Model How Correlation Occurs
Coefficients are Calculated (2)
Risk Probability = .25,
Range .8, .95, 1.05
Risk Probability = .5,
Range .95, 1.05, 1.15
Activity 1
Risk Probability = .45,
Range 1.0, 1.10, 1.20
Activity 2
Correlation = 37%
Correlation is modeled as it is caused in the project
Correlation coefficients are generated, not guessed
© 2013 Hulett & Associates, LLC
21
Scatter showing 37% Correlation
© 2013 Hulett & Associates, LLC
22
Sensitivity to the Risk Drivers
Risk 1 is more
important since it
affects both Activity
A and Activity B
© 2013 Hulett & Associates, LLC
23
http://www.gao.gov/products/GAO-12-120G
© 2013 Hulett & Associates, LLC
24
Schedule Check Report in Pertmaster
©(C)
2013
Hulett &Hulett
Associates,
LLC
2010-2013
&
25
25
Using Acumen FUSE for Best Practices
©(C)
2013
Hulett &Hulett
Associates,
LLC
2010-2013
&
26
26
Simple 2-Stage Space Vehicle Schedule
Software used: Pertmaster v. 8.7
© 2013 Hulett & Associates, LLC
27
Two Types of Risk

Inherent variability including duration estimating error –
uncertainty – Probability = 100%
– Used Quick Risk of -5% and +10%, could be reference ranges for
–

different types of activities
Could use reference ranges that would differ by type of activity
Discrete risks derived from Risk Register – Probability < 100%
– Summarized from detailed Risk Register
– These have a probability of occurring and an impact on specific
–
activities if they do
Parallel to their Risk Register information
© 2013 Hulett & Associates, LLC
28
Standard 3-point Range Representing Inherent
Variability and Duration Estimating Error
Inherent variability
and estimating
error: Optimistic 5% Pessimistic
+10%
© 2013 Hulett & Associates, LLC
29
Results with Inherent Variability and
Duration Estimating Error Only
Spacecraft for PMChallenge 2009
Entire Plan : Finish Date
100% 30 Oct 20
95% 27 Aug 20
400
90% 14 Aug 20
85% 06 Aug 20
350
Deterministic: 13 APR
2020 is 1%
80% 30 Jul 20
75% 24 Jul 20
300
70% 20 Jul 20
60% 10 Jul 20
250
Hits
55% 06 Jul 20
50% 01 Jul 20
200
45% 26 Jun 20
40% 22 Jun 20
150
Cumulative Frequency
65% 15 Jul 20
35% 17 Jun 20
30% 11 Jun 20
25% 08 Jun 20
100
20% 01 Jun 20
P-80 is 30 JUL 20, about
3.5 months later than
planned
Spread from P-5 to P-95
is 5 MAY 20 to 27 AUG
20 for 3.7 months
15% 25 May 20
50
10% 15 May 20
5% 05 May 20
0
0% 11 Mar 20
15 Mar 20
04 May 20
23 Jun 20
12 Aug 20
01 Oct 20
Distribution (start of interval)
© 2013 Hulett & Associates, LLC
30
Risk Analysis on Space Vehicle Project
Risk Drivers are from Risk Register
Risk
Requirements have not been decided
Several alternative designs considered
New designs not yet proven
Fabrication requires new materials
Lost know-how since last full spacecraft
Funding from Congress is problematic
Schedule for testing is aggressive



Min
Most Likely
95%
105%
95%
100%
90%
103%
95%
105%
100%
100%
90%
105%
100%
120%
Max
Likelihood
120%
70%
115%
60%
112%
40%
115%
50%
105%
30%
115%
70%
130%
100%
Seven risk Drivers have been identified and quantified.
Each Risk has probability assigned
Five have optimistic ranges possible, two are pure threats
© 2013 Hulett & Associates, LLC
31
Risks Assigned to Activities (1)
Risk
Requirements Not
Complete
Alternative Designs
Possible
Requirements
Definition
FS Preliminary
Design
FS Final
Design
FS
Fabrication
X
X
Designs Not Proven
X
New Materials in
Fabrication
X
Lost Know-How
X
Funding Problematic
Test FS
Engine
X
X
X
Testing Schedule
Aggressive
X
X
© 2013 Hulett & Associates, LLC
32
32
Risks Assigned to Activities (2)
Risk
US Preliminary
Design
US Final
Design
US
Fabrication
US
Test
Integration
Integration
Testing
Requirements Not
Complete
Alternative Designs
Possible
X
Designs Not Proven
X
New Materials in
Fabrication
X
Lost Know-How
X
Funding Problematic
X
X
X
Testing Schedule
Aggressive
X
X
X
X
© 2013 Hulett & Associates, LLC
X
X
33
33
Results Adding
Risk Drivers to the Background Risk
Spacecraft for PMChallenge 2009
Entire Plan : Finish Date
100% 08 Mar 22
95% 06 Aug 21
550
90% 16 Jun 21
Baseline 13 APR 20 is only
3% likely
85% 10 May 21
500
80% 07 Apr 21
450
75% 03 Mar 21
70% 05 Feb 21
80th percentile is 7 APR 21,
11.8 months later
60% 22 Dec 20
350
55% 03 Dec 20
300
50% 16 Nov 20
45% 29 Oct 20
250
40% 13 Oct 20
35% 28 Sep 20
200
30% 11 Sep 20
150
25% 31 Aug 20
20% 14 Aug 20
100
15% 28 Jul 20
10% 30 Jun 20
50
5% 13 May 20
0
0% 12 Nov 19
25 Jan 20
12 Aug 20
28 Feb 21
16 Sep 21
Distribution (start of interval)
© 2013 Hulett & Associates, LLC
34
Cumulative Frequency
65% 13 Jan 21
Hits
Spread P-5 to P-95 is
13May20 to 6 Aug 21, for ~
15 months
400
Activity Tornado Chart
from All-In Simulation
Spacecraft for PMChallenge 2009
Duration Sensitivity: Entire Plan - All tasks
00025 - US Fabrication
81%
00011 - FS Fabrication
81%
00028 - Integration
79%
00023 - US Final Design
76%
00009 - FS Final Design
75%
00021 - US Preliminary Design
70%
00007 - FS Preliminary Design
70%
00029 - Integration Testing
Risky Activities:
Fabrication, Integration,
Final Design, Preliminary
Design, Testing
All are correlated with the
finish date > 60%
These are activities /
paths, NOT RISKS
63%
00026 - US Test
61%
00012 - Test FS Engine
61%
© 2013 Hulett & Associates, LLC
35
Risk Driver Tornado
from All-In Simulation
Risk Factors Driving Project Schedule
6 - Funding from Congress is problematic
4 - Fabricaton requires new materials
3 - New designs not yet proven
The main RISK, however, is
funding from Congress,
which affected all activities in
this model.
7 - Schedule for testing is aggressive
2 - Several alternative designs considered
5 - Lost know-how since last full spacecraft
This is the main risk to
mitigate, if possible
1 - Requirements have not been decided
8 - Cost Risk is based on immature data
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
Correlation
© 2013 Hulett & Associates, LLC
36
Contribution of
Each Risk to the Contingency (1)
Explain the Contingency to the P-80
P-80 Date
All Risks In
Take Risks Out:
7-Apr-21
Days Saved
% of Contingency
Specific Risks Taken Out in Order
23-Nov-20
135
Testing Schedule
5-Oct-20
49
New Materials
3-Sep-20
32
Alternative Design
21-Aug-20
13
Requirements
14-Aug-20
7
New Design
6-Aug-20
8
Lost Know How
31-Jul-20
6
38%
14%
9%
4%
2%
2%
2%
Uncertainty
Natural Variation &
Estimating Error
Total Contingency
13-Apr-20
109
30%
359
100%
Funding
© 2013 Hulett & Associates, LLC
37
Contribution of
Each Risk to the Contingency (2)
100%
90%
Variation:108
Variation:6 :8n:7 n:13 ation:32
Variation:49
Variation:134
80%
60%
50%
40%
30%
Variation:50 ariation:3 4 5 3 :11 ariation:45 ia ion:1
20%
10%
0%
19
20 Jan 20
10 Mar 20
29 Apr 20
18 Jun 20
07 Aug 20
26 Sep 20
15 Nov 20
04 Jan 21
23 Feb 21
14 Apr 21
03 Jun 21
© 2013 Hulett & Associates, LLC
23 Jul 21
11 Sep 21
31 Oct 21
20 Dec 21
08 Feb 22
38
Cumulative Probability
Graphic of the effect
of taking the risks
out in order of
priority at the P-80
70%
Mitigating the Most Important Risk
Effect of Partially Mitigating the Risk with the Highest Priority
Min
Funding from Congress is problematic
90%
Most Likely
105%
Max
115%
Likelihood P-80 Date
70%
7-Apr-21
Mitigation: Situate your suppliers strategically by Congressional District of the members of
the Committee. Make the “jobs” case for continuous funding.
Impact on the parameters
90%
105%
115%
30%
5-Jan-21
Mitigating the risk is estimated to reduce the probability from 70% to 30%
but, if the risk happens, the impact range remains the same. This saves
92 calendar days at the P-80 target level of confidence
© 2013 Hulett & Associates, LLC
39
Summary (1)





The focus is on the risks, not their impact
Risks “explain” the need for a contingency
Management appreciates this focus on risks
Risk interviews are conducted at 5,000 foot level, where
people typically think of risk
Interviews go faster, stick to the substance
© 2013 Hulett & Associates, LLC
40
Summary (2)


Use Risk Register for quantitative analysis
Specific risks can be quantified and assigned to schedule
activities
– Quantification is probability and impact
– A risk can affect several activities
– An activity can be affected by several risks

Risk Drivers can be combined with other more traditional
approaches such as 3-point estimates for inherent variability
risk, estimating error or for probabilistic branching
© 2013 Hulett & Associates, LLC
41
The Risk Driver Approach to Project
Schedule Risk Analysis
A Webinar presented by
David T. Hulett, Ph.D.
Hulett & Associates, LLC
To the
College of Performance Management
April 18, 2013
© 2013 Hulett & Associates, LLC
42