Analysis, Modeling & Simulation
Joe Adamo
Department Head - Operations Analysis
Joyce Wheeler
Manager – Operations Analysis
This document does not contain technical data as defined in the International Traffic in
Arms Regulations (22 CFR 120.10) or the Export Arms Regulations (15 CFR 779.1).
Agenda

Intro to Operations Analysis

What is Analysis, Modeling, and Simulation (AM&S)?

Case Studies

Summary

Questions & Discussion
Operations Analysis
Operations Analysis is the analytical arm of the
Systems Engineering discipline. It is the
discipline of applying advanced analytical
techniques and processes to provide information
for better decision making.
Who Does Operations Analysis?

Operations Analysts come from a variety of disciplines







Aeronautical Engineering
Mechanical Engineering
Electrical Engineering
Computer Science and Engineering
Mathematics, Physics
Operations Research
Key Traits are Analytical Skills, Problem-Solving Capability, and
Communication Skills
Degree
Masters
53%
PhD
5%
BS/BA
42%
Approximately 600
Operations Analysts
in Boeing IDS
What Do Operations Analysts Do?
An Aircraft Example:

The Realm of Designers – Measures of Performance




Range
Speed
Payload
The Realm of Ops Analysts – Measures of Effectiveness




Survivability
Number of Targets Detected
Lethality – Number of Targets Destroyed
Number of Information Packets Sent/Received
and Cost/Affordability
Ops Analysts Work with the Design Team to Ensure
the System Can Do It’s Intended Job
Tools and Methods
 Commonly Used Techniques
 Modeling and Simulation
 Statistical methods
 Linear and nonlinear programming
 Econometric methods
 Queuing and other stochastic-
 Decision analysis
process models
 Mathematical Modeling - Nearly all of these
techniques involve the construction of a model
that attempts to capture the behavior of the
system being studied
Analysis Modeling and Simulation Defined

Analysis: Evaluating and assessing system
performance, processes, and relationships to determine
measures of importance such as effectiveness,
performance, and reliability; typically using the results
from models and simulations

Modeling: Creating representations (models) of
systems including their processes, behaviors, and
interactions

Simulation: Using (executing) models over time to
investigate system performance
Model vs. Simulation
 Model - Representation of the physical
entity
Example: Static Model of the Earth
 Simulation - Execution of model over time
Example: Simulating the Earth’s Rotation
Modeling and Simulation
 Allows the analyst to conduct trade studies and
answer “What if?” questions
 Can assign values to the different model components
and clarify the relationships among them
 Values can be altered to examine what may happen to
the system under different circumstances.
 Inputs are modified and the model re-run to identify
the best of levels for variables or combinations of
design features
Spectrum of Models and Simulations
Interactive Digital
Simulation
Virtual
Environment
Constructive
Simulation
Voice of
the
Customer
Decision
Analysis
IWARS
Requirements Definition
Customer Involvement
Management Understanding
Quick Tools
Customer-approved
simulations
Workload Assessment
Selected Controls/
Displays
Pilot in the Loop
Benefits of AM&S
85% of Cost Committed
15% Cost Expended
100
Percent of Life-Cycle
Cost Committed
Percent
90
85
80
95
70
70
60
50
40
30
20
30
Brief Window of Opportunity
10
5
0
Concept
Refinemt.
Tech
Develop.
System Develop
and Demonstration
Percent of Program
Cost Expended
Production and
Operations and Support
Product Life Cycle Phases
Early Emphasis on Analysis Prevents Costly Design
Changes Later in The Development Cycle
Demo
CheapFoods Distribution
Center
CheapFoods Distribution Center
 CheapFood Supermarkets has several stores in
the St. Louis area.
 However, each is supplied from a different
warehouse out of state.
 To cut costs, the owner of CheapFoods has
decided to build a warehouse in the St. Louis area.
CheapFoods Products
CheapFoods only sells three items. However, each has
to travel on its own truck because of certain
constraints.
Item Requirements:
3
Supply Cost:
Ice Cream
Requires freezer truck.
3x$
2
Meat
Requires refrigerator
truck and is perishable.
2x$
1
Twinkies
Doesn’t require special
truck and lasts forever.
1x$
CheapFoods Territory
Problem Solved!
Great! Now, supply costs for the supermarkets are at a
minimum.
The owner can go home and buy that Ferrari he’s been
wanting and the operations engineer can go back to his
desk.
This was a VERY simplified example but it shows how a model can
help provide insight into problem and identify a solution
When Do You Use AM&S?
Product Life Cycle
Customer
Needs
Analysis
Define Mission/
Function
Requirements
Define System
Requirements &
Concepts
Define/
Develop
Concept
Perform
Perform
Preliminary Detailed
Definition Definition
Production
Support
Voice of the Customer / Needs Analysis
Requirements Analysis / Trade Studies
Cost & Affordability
Effectiveness Analysis / Trade Studies
Analysis of Alternatives
Advanced Supportability
Impact of Technologies
Create New
Competitive Assessment
Capture New Test & Evaluation
Business
Business
Keep It Sold
AM&S is Needed Across the Product Life Cycle
Customer Needs Analysis /
Requirements Definition
Decision Analysis
Competing
Alternatives
Multiple
Suppliers
Multiple
Disciplines
Conflicting
Interests
en
te
d
Desires
Do
cu
m
Wants
Needs
De
fen
sib
le
Must Haves
Wishes
Decision Analysis Techniques Are Tools
Used to Solve Complex Problems
Through a Structured Process
Co
mm
un
D
ica
eci
Dis
tio
s
cip
i
o
n
nA
line
d
nal
ysi
s
Multiple
Customers
Multiple
Objectives
Decision
DecisionAnalysis
AnalysisProvides
ProvidesMore
MoreCustomer
Customer
Interaction
and
a
Better
Product
Interaction and a Better Product
Understand Customer Needs
 Operational Requirements
 Budget constraints
 Network Environment (FAA, DOD,
etc.)
Foc
us
Consensus
Common Terminology
List of Potential Trades
Priorities
Documentation
Begin long term Customer Relationship
 Deriving Requirements
 Developing plans for product design
 Ensuring Customer involvement
throughout product life cycle
Cost-Effectiveness Trades
Affordability
Unmann ed C ombat Air V ehicl e
Phase II - Affordabi lity / LCC P lan
Pr e pa red by :
C oncurr ed by :
A pp ro ved b y:
David M cC aughey (B oeing)
Kurt Bau sch (B oeing)
Phil Panagos (B oeing)
S teve Ras t(S AI C)
Lt Col Michael Leah y(USAF)
$
Time
Affordability
Plan
Focus:
$
Time
• System Cost
Drivers
• Figures of Merit
• Effectiveness &
Affordability
Balance
$
Time
$
Point
Estimates
$
Historical
Regression
Cost
Uncertainty
Simulation
Cost
Targets
Supplier
Options
LCC Probability
100%
80%
60%
EMD
Prod
O&S
40%
20%
$
0%
F-16
JSF
UCAV
UCAV
75% Reduction from F-16
1/3 the cost of JSF
1998
1999
1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q
2000
Time
High
Med
Risk Factor
Development /
Investment Plans
2002
2003
UCAV ATD
Phase II
Establish Common Avionics
Development Group
Deliver B-2 weapon
delivery GWIS
DEMPC
Global theater multi-level networking demo
UCAV decision aids lab demo
(contingency management)
BOLDSTROKE demos
UCAV decision aids flight demo
Loss of comm contingency flight demo
Primary: Fully integrated
software functionality.
Fallback: Decreased software
functionality.
UCAV ATD RR&OE
Gov’t S&T
Boeing
UTP
Suppliers
}
Phase II Start
Last revision:
2004
RR&OE
Real-time software architecture
& design demo
Single simulated
vehicle distributed
control lab demo
UCAV ATD Phase I
UCAV ATD Phase II
Low
$
2001
2005
1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q
UCAV ATD
Phase I
• Investment
Planning
$
O&S Cost
UCAV - A TD
Unit Cost
Advanced Technology Demonstration
EMD
Real-time distributed processing
AJ/LPI LOS C2 Demo
UDS Formation Taxi/Flight (fixed geom, pos sep algorithms)
Software reuse metrics tracking
Lab & flight demo - OMP/mission/vehicle
systems integration
AT3 or PLAID test on UCAV
JSF/UCAV Commonality Study
Supplier software productivity demo
Software reuse metrics tracking
AJ/LPI BLOS C2 AJ GPS Demo
Drop multiple pre-planned small
SAR flight test
bombs from MBR with full SMS
on UCAV
Decision aids for
operator handoff
lab demo
Demo of OMP & Intelligent
Maintenance Aids /
PMT & IMSS
Automated dynamic mission
replanning flight test demo
Multi-sensor multiUDS Coordinated motion, variable
source data fusion
geometries / deconfliction algorithms
PHM/OMP
UDAS Algorithmic Control Flight Test Demo
(Multi-Vehicle Coordinated Flight, Collision
Air traffic mgt demo
Avoidance, Sensor Planner, Autorouter)
Flt Demo 1
Flt Demo 2
Flt Demo 3
Flt Demo 4
Flt Demo 5
RR&OE Start
Phase II End
EMD Start
Kurt Bausch
314-232-6917
 Life Cycle Cost/Total Ownership Cost
 Design to Cost
 Best Value
Analysis of Alternatives

Work with customer to identify design, configuration, or
approach alternatives

Evaluate operational effectiveness of each alternative

Evaluate cost considerations for each alternative

Compare cost and performance for each alternative

Provide customers with insight to desired options

Provide justification and support for selecting the
preferred alternative
National Air Space - Wide Delay /
Capacity Analysis for
BCA Flight Operations Strategy
Analysis of Alternatives
Statement of Problem and Alternatives
•
Quantify, in terms of reduced system delays, the
benefits of the alternative airport capacityincreasing concepts
 Alternative 1: Navigation improvements
 Alternative 2: Alternative 1 plus landing system
improvements
 Alternative 3: Alternative 2 plus Air Traffic Control (ATC)
improvements
Modeling Approach
Boeing National Flow Model (NFM)
 National Air Space (NAS)-wide traffic analysis for a
single day
 Simulation with a network of queues
 Queues for capacitated elements
 Input flight schedules and capacities
 Output delay statistics
 Directly addressed delay propagation
NFM Network of
Queues
Assumptions



Used Current Market Outlook-based future schedule
generation capability to analyze traffic levels for y2000,
2010, 2015, and 2020
Annualized results by taking the weighted average of six
“representative” days
Assumed no airline schedule re-planning, including no
flight cancellations
Conclusion and Recommendations


Successfully supported benefits analysis comparing the relative effectiveness of
the alternatives
ATC improvements (associated with Alt 3) were shown to have the greatest
benefit
The 20-25% capacity increases of Alt 3 are magnified into a 50% reduction in
delays when confronted with y2020 traffic
80
70
Better
Ave Arrival Delay (min)

60
50
baseline
alt1
40
atl2
30
alt3
20
10
0
2000
2010
2015
Year
2020
Benefits to Stakeholders



Methodology can be used to show how capacity increases
translate to end-user benefits in terms of decreased delays
Methodology is more credible than alternative approaches because
it considers delay propagation, airport inter-dependencies and
weather conditions correlated between airports
Helps direct executive-level decision making in terms of
which alternatives are most cost-effective to pursue



Helps BCA to price avionics products
Helps influence air traffic service providers to improve system
performance so Boeing can sell more airplanes
Provides analytical evidence of benefits that may be used in
presentations to airline customers
Advanced Supportability AM&S
•
•
•
•
•
•
•
•
Prognostics & Health Management
Opportunistic maintenance
Interactive Tech Manuals
Proactive manufacture
Proactive supply
Autonomic distribution
Spares usage & trends
Projected spares needs
Repair
&
Overhaul
POL
Pipelines
• Flexible support
COD
Resupply
• 24/7 Response centers
• Digital Engineering Links
 Supply Chain
Management
Intermediate
Repair
OEM
 Maintenance
Management
Analysis
Aircraft
Vehicle
Equipment
Generation &
Maintenance
Ground
Deliveries
 Supply
Management
Analysis
NASA Cargo Processing for the
Shuttle and Space Station
Multi-Purpose Logistics Module
(MPLM)
Assembly Process
Statement of Problem



With the elements of the International Space
Station being delivered to Earth orbit, Multiple
cargo missions were needed
NASA experienced complex delays and problems
in loading the MPLM and processing cargo for
the Shuttle missions
Needed to find the problem areas and test
potential solutions
Potential Alternative Solutions

Current process had not yet impacted a
Shuttle launch date




Excessive overtime and “last minute heroics” to
meet schedule
Either continue current process or find
alternative
Complete “re-write” of current procedures
and processes at high cost
Determine problem areas and test solutions
Modeling Approach




Developed model to replicate current processes
including current delay factors
Used Monte Carlo to replicate process
performance ranges
Showed sensitivity of each process step to
change and effect on overall process efficiency
Tested potential changes in process and
formulated course of action for correction
Conclusion and Recommendations




Determined a 60% increase in efficiency to meet
established schedules was possible
Identified process areas to target and tested possible
solutions
Replicated new process changes
Recommended four significant changes in cargo
processing – low cost with high return
Increased Efficiency Led to Boeing Win on Renewal of Cargo
Processing Contract at Kennedy Space Center
Advanced Supportability AM&S Benefits


Help identify, evaluate and integrate new technologies
to aid support concepts
Identify candidate new support options to improve
response time and reduce cost

Explore Sustainment options early

Identify required spares and staging requirements

Experiment with and predict support requirements

Ensure support options are well explored

Demonstrate to customer benefits of support plan
Test & Evaluation (T&E):
Virtual Integrated Simulation Network
Level 1: Systems Level
Flight Sim., Mesa, ITDL ,VWC
Huntsville, Philadelphia, CoS




Operators
AWACS
Canard Rotor Wing
Space Based Radar /
Space Assets
 Unmanned Combat Air
Vehicle / UAV
Level 2: Tactical Level
VWC






Concept of Operations
Systems-of-Systems
Operators
Interoperability
Tactical S&R/C2/BM\
Linked to FCS Joint
Virtual Battlescape
Level 3: Operational Level
BIC
 FAA
 Advanced NCO
Command & Control
 BMC4ISR
 Operators
 Interoperability
Virtual T&E Benefits

Work with customer to anticipate problems with design
parameters

Verify that detailed design meets customer requirements

Understand sensitivity to performance and cost

Anticipate problems before flight test

Predict Test & Evaluation performance

Minimize live test through virtual T & E and reduce
overall test and evaluation cost
F/A-18 Saved 155 Test Flights
through Judicious Use of Virtual T&E
Production Design AM&S
Process
Simulation
Paint Prep Facility Layout
Graphite Lay-up
Production Area
Production Design AM&S Benefits

Anticipate and correct Producibility problems

Model production flow

Identify bottlenecks


Identify recommended changes to production
line
Experiment with production improvements
without interfering with current production
schedule
This document does not contain technical data as defined in the International Traffic in Arms
Regulations (22 CFR 120.10) per 125.4(b)(13) or the Export Arms Regulations (15 CFR 779.1)
APPROVED FOR PUBLIC RELEASE 12/13/2004
GP42656027.ppt
AM&S Summary
Benefits of AM&S
 Economical method of testing a broad array of “What Ifs”
 Provides information to help balance Performance and
Cost
 Enables the demonstration of baseline capabilities under a
wide variety of environments/scenarios
 Provides the customer a consistent and transparent basis
for your decisions
 Strengthens the customers perception of how decision are
made at Boeing
Competitive Advantage




AM&S provides an early understanding of system
capabilities, performance, and effectiveness
AM&S provides information as to which design features
or system capabilities “buy” their way on to the system Cost as an Independent Variable (CAIV)
Simulation allows head to head comparison of
competitive designs before downselect
Simulation provides the customer with the opportunity
to experience how the product will perform
AM&S Allows the Customer to “See” How the System Will Perform
Questions?
Contact Info


Joe Adamo
314-233-2688
[email protected]
Joyce Wheeler
314-232-9322
[email protected]