The Value of
Technical Performance Measures
Ann Lynn
The Boeing Co.
(314) 234-9714
[email protected]
All Data in this pitch
is Notional – for example only
Agenda
• TPM Definition
• Typical Types of TPMs
• Constructing TPM Profiles
–
–
–
–
–
Performance Thresholds
Plan Over Time
Uncertainty
Management Process
Allocation of Margin
• Roll-up (Parent/Child) TPMs
• Value to a Program
Technical Performance Measures
• Represent critical technical thresholds and goals for
success of the program
• Should be variable and responsive to engineering
changes
• Usually exist in a hierarchy corresponding to the spec
levels (system, subsystem, component…)
• Have target values corresponding to specification values
– that is, your TPM should be a parameter in a spec at some level
• Are tracked and statused against their defined plan and
represent the technical health of the program
Typical Types of TPMs
• Parameters that flow from and/or support KPPs
• Key Constraints and Performance Reqmts
– Weight, power, heat load
– Reliability/ Maintenance parameters
– System / subsystem performance
•
•
•
•
•
•
Endurance
Range
Latency
Accuracy
Efficiency
Etc
– Cost (e.g. AUPC)
• Parameters associated with Program Technical Risks or
areas of technical challenge
TPMs should represent a balanced set of key parameters;
Don’t monitor something just because “you can”
Example TPM Status Chart
Example Monthly / Quarterly Status
Model
Type
Parameter
Spec
Status
0.66
Demonstrator
TPM Propulsion Efficiency
0.66
Demonstrator
Propulsion Max Take off thrust
TPM Vc
at 4400'
472
Demonstrator
TPM Propulsion Power Consumption (28V)
30
Demonstrator
TPM Propulsion Reliability
Demonstrator
TPM
(Meas/Model*)
497 lbs
(Meas/Model*)
30 Watts
(Estimated)
0.9969
0.9742
(Estimated)
^ Propulsion Weight
262
262 lbs
(Estimated)
Meas/Model*= Propeller Data Measured and used w/ Modeled Motor Data
This view shows: •Title/Definition
•Status Color (RYGB)
•Status against spec
•Maturity of the status
Definition of Performance Thresholds
(Generic)
• Blue – the system exceeds
requirements by more than x (and
may be subject to re-allocation)
TPM margin
Spec value
TPM margin
• Green – the system meets all
performance requirements
• Yellow – the system can meet some
objectives but does not meet all
performance requirements
• Red – the system cannot perform
satisfactorily
Note, reqmts can be negotiated/updated with
corresponding updates to Specs and TPMs.
Setup as shown, the parameter has to meet the spec to be green.
TPM Profile Over Time
This view of the TPM allows you to see history and trends in performance
Profiles over time (cont.)
26
25
24
23
%
22
TPM
Spec
Status
Unit Efficiency
24%
22.2%
(Meas.)
21
20
TPM using current status, not
predicted status at end of program
19
18
Fail
Warn
Plan
Goal
Status
Spec Value
Upper
For this TPM, performance is planned to improve over time.
If actual progress >= planned, you are green, even if you haven’t achieved
the end-state target (spec value)
Weight Management
SR#4
FD 3+
SR#8
FD 4A
SR#9
FD 4B
SR#10
FD 4B
Projected weight (projected
value at end of program)
includes a historical growth
factor
Current
Projected Wt.
(w/ Growth Risk)
3971.3 lb
Spec
NTE Weight
3850 lb
IPT Weight
Allocations
3588 lb
Current
Baseline Wt.
3588.0 lb
VTR
PDR
J-11 A-11 S-11 O-11 N-11 D-11 J-12 F-12 M-12 A-12
Date
• Baseline weight reflects
current configuration sized
to loads at NTE weight
requirement.
TPM if using current status
TPM
Spec
Status
System Weight
3850
3588
(current)
TPM if using projected status
TPM
Spec
Status
System Weight
3850
3971
• Projected weight includes
planned design changes and
X% weight maturity growth
risk for post PDR design
maturity.
(projected)
Weight TPM usually plotted per the Weight Mgmt process;
Show both values, status to Projected;
9
TPM Profile with uncertainty bar
include an “uncertainty” bar
for each reported value
Spec value line
If your project has significant Risk, or it is early in the project,
TPM status can be deceiving unless uncertainty is understood
TPM Management Process
(Generic)
In the blue: consider adjusting that
parameter’s spec value and taking
relief elsewhere
In the green: press on!
TPM margin
Spec value
TPM margin
In the yellow, or alarming trend: take a
program-defined action (put on Risk
Watch list, initiate reqmts/design trade,
establish new risk mitigation, initiate
change to design, reqmts, or plan)
In the red: take a program-defined action
(establish new risk mitigation, initiate
change to design, reqmts, or plan)
The approach to monitor and control is predicated on
correct allocations of margin
Margin
Definition of TPM Margins
TPM margin
Spec value
TPM margin
Factors to consider:
1. Margin may be allocated based on
the amount of risk (e.g. Red Risk
or low TRL subsystem gets a
greater share of margin)
2. Margin may be allocated based on
relative impact to System
performance (accounts for
sensitivities)
3. Margin may be limited by the “hard
points” - those parameters that
have a hard stop which prevents
acceptable operation (e.g.
structural limits for weight)
Margin: How bad can it get before status goes red?
How good must it be before status goes blue?
*System Parameter* Tree
*System Parameter*
Power Consumption
Energy Collection
SPC efficiency
Energy Storage
Propulsion Consumption
Prop efficiency
Aero efficiency
ESS RT efficiency
ESS degradation
Subsys Pwr Consump
VMS / MS
AV Weight
Cruise consump
Subsys Z
VMS
maneuvers
Drag
Subsys Z
ESS components
A components
Stability
ESS
B components
ESS param
Power Distribution
EPD efficiency
PPX efficiency
Subsys A
Subsys B
Propulsion
Subsys X
Convertor efficiency
Wiring efficiency
High Sensitivity
This diagram maps the subsystem parameters
that contribute to the system-level parameter;
Sensitivities are noted.
Lower sensitivities
Associated Risks
*Sys Parameter*
Power Consumption
Energy Collection
46-xxx
SPC efficiency
Propulsion Consumption
Avionics & PSS Consump
24-xxx
Energy Storage
1-xxx
Prop efficiency
Aero efficiency
ESS RT efficiency
VMS
AV Weight
Cruise consump
Subsys Z
57-xxxx
VMS
maneuvers
54-xxx
41 - xxx
7-xxx
ESS degradation
Power Distribution
2-xxx
EPD efficiency
56-xxxx
PPX efficiency
Convertor efficiency
Drag
Subsys Z
51-xxx
Stability
50-xxx
ESS
18-xxx
6-xxx
ESS components
A components
B components
ESS param
Subsys A
Subsys B
Propulsion
Shows Risk #, Title,
and current level
(RYG)
Subsys X
56-xxxx
Subsys x param
High Sensitivity
Lower sensitivities
Wiring efficiency
This diagram maps Risk to
the subsystem parameters.
Helps show where margin is needed.
Interdependent TPMs
• Competing for their share of the overall margin
• Their performance needs to be monitored as a
group
• Change (re-allocation of margin, spec
adjustment) has to be managed for the group
KPPs and TPMs
KPPs
Time of Year
at which Energy balance
is closed
TPMs
*System Parameter*
System
level
(AV Efficiency
at End of Mission,
AV Performance)
SubSystem
&
Cmpnt
level
-Propulsion eff
-ESS round trip eff
-PPX eff
-SPC eff
-AV Aero eff
AV Weight
AV Power Consumption
-Propulsion Power consump
-VMS Power consump
-Airframe Power consump
-ESS Power consump
-Subsys A Power consump
-Subsys Z Power consump
-VMS weight
-Propulsion weight
-Airframe weight
-ESS weight
-PPX weight
-Subsys A weight
-Subsys Z weight
KPPs represent customer operational objectives;
TPMs shown represent measurable design parameters that contribute to the KPP
Relationship of System and
subsystem TPMs
• Example: Command
Latency <= 160 ms
Weapon Lnchr
25
5
Weapon
Spec =
20 ms
SMS
25
MC
25
Launcher
Spec =
5 ms
• If supplier specs are
more stringent, that
PVI
means margin is held at
50
30
subsystem level above
supplier
PVI CMPNT1 • Spreadsheets can help
Spec =
account for values and
35 ms
margins at each level
• Many PMs prefer to
PVI CMPNT2 monitor Supplier-owned
TPMs when available
Spec =
System
margin
15 ms
Example parent/child TPMs (1 of 3)
Owner
TPM Title
Spec
Value
Current
Status
CE
System Command Latency
160 ms
136 ms
Dsplys
^PVI Cmd Latency
50 ms
50 ms
Msn
Cmptr
^MC Cmd Latency
25 ms
25 ms
Stores
^SMS Cmd Latency
25 ms
25 ms
Stores
^Lnchr Cmd Latency
5 ms
5 ms
Stores
^Weapon Cmd Latency
25 ms
31 ms
System TPM can be green even when child (subsystem) TPM is red;
in this case because of margin held at system level
Example parent/child TPMs (2 of 3)
Owner
TPM Title
Spec
Value
Current
Status
CE
System Command Latency
160 ms
121 ms
Dsplys
^PVI Cmd Latency
50 ms
35 ms
Msn
Cmptr
^MC Cmd Latency
25 ms
25 ms
Stores
^SMS Cmd Latency
25 ms
25 ms
Stores
^Lnchr Cmd Latency
5 ms
5 ms
Stores
^Weapon Cmd Latency
25 ms
31 ms
System TPM status can be green if child (subsystem) TPMs balance out
Example parent/child TPMs (3 of 3)
• Example: Command
Latency <= 160 ms
Weapon Lnchr
25
5
35
Weapon
Spec =
25 ms
35
SMS
25
MC
25
PVI
50
System
margin
30
20
Launcher
Spec =
5 ms
Over time, System TPM margin may need to be re-allocated
to compensate for subsystem over target
Subsystem TPM in the blue range may have spec adjusted and
“give back” margin to the System level or to another subsystem
TPMs - Takeaways
 Create a balanced set of key parameters
 Ensure the full Profile is understood by program
technical leadership, even if only the stoplight format is
used for periodic reviews
 Profile over time current vs projected status
 Basis for margin  maturity / uncertainty
 Assess the risk and sensitivity of parameters to support
appropriate margin allocation
 Know where your margin is … and isn’t, so you can react
to changes in reqmts or in design status
 TPMs can be used as a predictor of success … and
failure
The Value of TPMs
• Track the “right” things
– Make sure your TPMs correlate to key technical
parameters
• Track technical progress compared to planned
– Your monitoring system is only as good as the plan &
thresholds you set it up with
• Spend the time upfront to set up correctly!
– Your monitoring system is no more accurate than the
data you put in
• maturity / uncertainty of reported data must be evaluated
• Conisder plotting uncertainty bars on data points to avoid
false sense of security