Significant Seven & Beyond
Laura Madden
UK Civil Aviation Authority (CAA)
Safety Performance Analyst &
Bowtie Programme Manager
28th September 2015 1
Objective
To share a Regulator’s Bowtie journey and our
Bowtie developments
Content
- Introduction to CAA and our remit
- CAA’s Bowtie History
- Significant Seven Template Project
- After Significant Seven Project
- Bowtie Strategy and focus groups
- Safety Performance Indicators
- Risk Assessment
- Oversight
- Other Projects
2
The UK Civil Aviation Authority (CAA)

The CAA is the UK's specialist aviation regulator. Its regulatory
activities range from making sure that the aviation industry meets the
highest technical and operational safety standards to preventing
holidaymakers from being stranded abroad or losing money because
of tour operator insolvency.

Safety and Airspace Regulation Group (Gatwick)
 Intelligence, Strategy and Policy (ISP)
 Intelligence: Ensure the right processes and data are in place
to:
•
•
•
•
Identify safety issues
Assess and prioritise their risk
Measure safety performance
Promote the results so the right actions can be taken to improve safety
3
The CAA’s Bowtie History
 Got
hooked around four years ago
 Saw
potential and use
 Risk
Management moving forwards based on Dr James
Reason’s Swiss Cheese Model
 Picture
and story telling
 Small
internal testing to understand use and worked with
two key UK organisations who already used bowtie
 Share
the information and innovative way of displaying and
understanding risk
4
Where we started…
 Accident
outcomes, but how do they manifest themselves,
what causes them, what controls are in place to mitigate
their risk and how effective are they?
Runway Incursion
Runway Excursion
Fire
Airborne Conflict
CFIT
Ground Handling
Loss of Control
5
Significant Seven Bowtie Template Project
 Templates
focused on issues relevant to each Sig 7 item

Created by multi-disciplinary groups of CAA and industry
experts
 Focus on large commercial air transport operations
 Total
system approach (including third party risks)
 Freely
available with guidance for industry to tailor to their
own operation and use within their SMS
 www.caa.co.uk/bowtie
 For
the regulator:

Support performance-based oversight
 Data collection and safety performance monitoring
 International engagement
6
The Bowtie Templates
 Core
Bowties (7)

Prominent precursor events (identified through data)
 Fully completed with all elements and additional information
 Supplementary
Bowties (14)

Other precursor events
 Skeleton bowties
 Umbrella
Bowties (3)

Generic issues independent of event
 Human – Technical - Environment
7
Appreciation for Controls
Criticality
Criticality
Threat Exposure
Safety Performance Indicator (SPI)
Document
LinkOwnership
Control
Functionality (brf
Effectiveness
codes)
Control Type
8
The Bowtie Templates
Sig 7
LoC
LoC
LoC
RE
RE
RE
CFIT
CFIT
CFIT
RI
RI
RI
AbC
AbC
AbC
GH
GH
GH
FIRE
FIRE
FIRE
Hazard
Operation of Large CAT Fixed wing aircraft (human performance)
Operation of Large CAT Fixed wing aircraft in adverse environmental conditions
Operation of Large CAT Fixed wing aircraft experiencing technical failures
Large CAT Fixed wing aircraft - Landing Operations
Large CAT Fixed wing aircraft - Take-off and Landing Operations
Large CAT Fixed wing aircraft - Take-off/Departure Operations
Large CAT Fixed wing aircraft - Non-precision approach (especially in IMC or at night)
Large CAT fixed wing aircraft - Arrival or departure (general)
Large CAT fixed wing aircraft - Precision approach in IMC or at night
Large CAT fixed wing aircraft operating on the ground in or close to the protected area of an active runway
Ground vehicles and/or people in or close to the protected area of an active runway for large CAT aircraft
Large CAT Fixed wing aircraft - Landing Operations
Two or more aircraft in vicinity of each other whilst airborne in UK Class A airspace
CAT en-route operations in UK Class G airspace
CAT operations in the circuit area of a procedural controlled aerodrome
Loading operations for large CAT fixed wing aircraft at UK Aerodromes
Ground vehicle operations in the vicinity of large CAT fixed wing aircraft
Cold weather conditions conducive to large CAT fixed wing aircraft contamination (on the ground)
Aircraft electrical systems on large CAT fixed wing aircraft
Combustible materials in the cargo compartment on large CAT fixed wing aircraft
Fuel and combustible aircraft components for large CAT fixed wing aircraft - external to the pressurised areas
Top Event
Aircraft unintentionally deviates from normal in-flight parameters (aircraft upset)
Aircraft unintentionally deviates from in-flight parameters (aircraft upset)
Aircraft unintentionally deviates from in-flight parameters (aircraft upset)
Inability to make a stop within the expected landing distance required
Loss of directional control on the runway
Inability to become airborne within the expected takeoff distance requirement
Terrain separation deteriorating below normal requirements
Terrain separation deteriorating below normal requirements
Terrain separation deteriorating below normal requirements
Incorrect presence of aircraft in the protected area
Incorrect presence of vehicle or person in the protected area
Incorrect presence of aircraft in the protected area
Two or more aircraft are in such proximity that their safety is or may be compromised
Two or more aircraft are in such proximity that their safety is or may be compromised
Two or more aircraft are in such proximity that their safety is or may be compromised
Aircraft significantly outside the operational mass and balance envelope
Significant ground damage undetected prior to aircraft commencing takeoff
Aircraft commences takeoff with contaminated flying surfaces or engines
Hidden area fire becomes established
Cargo compartment smoke/fire
Uncontained fire occurring external to the pressurised areas of the aircraft in-flight
9
Post Project
 Large
uptake, worldwide, which has exceeded expectations
including airlines, aerodromes and other regulators
 Implementation
projects underway with UK aviation
organisations

To appreciate the feasibility, practicality and benefit of
embedding bowties within an organisation’s SMS
 Training
of CAA staff
 Creation
of a joint regulator-industry Bowtie User Group
 Endorsement
of CAA Bowtie Strategy
10
SAFETY REPORTING &
CLASSIFICATION
INTERFACES
Problem Solving
on Hot Topics
Risk Classification
Constructing specific
bowties to assess a
particular
issue/event
Identifying high level
controls to allocate
risk grading to safety
events
Safety Event
Reporting
Using the controls to
improve quality of
safety reports
Performance
Based Oversight
Audits
SSP / SMS
Hazard
Identification
Identifying threats and
top events in the safety
system
Risk Assessment
Qualitative
assessment of
controls
Focus on under
performing controls
on oversight audits
Debriefing Aid
Risk Mitigation
Running safety
events through the
bowtie to appreciate
the severity
Incident/ Accident
Investigation
Total system
approach considered
during an
investigation
Taking actions to
improve the controls
Critique of Industry
BowTies
To review practicality and
whether critical controls
have been considered
Safety Promotion
Safety Assurance
Raising awareness of
controls in the safety
system
Identify and monitor
SPIs to measure
success of the actions
CAA Bowtie Strategy
CAP1329

Purpose




To maximise the use of bowtie
CAA and industry
Inform decision making so that the right actions can be taken to
achieve the best safety outcomes for the UK public
Objectives





Known as the world leader on aviation bowtie risk modelling
expertise
Embed methodology and its output within Regulatory SMS to help
drive safety improvement activity and safety planning
Output is fundamental to Performance Based Oversight (PBO)
Improve our intelligence capability identifying and driving data
collection
Raise aviation safety management standards
12
Safety Performance Indicators
what matters in our safety system – measuring
control effectiveness
 Appreciate
BowTie
Element
Type
SPI Description
Top Event
Threat
Crit Ctrl
Crit Ctrl
Audience Activity or
Level
Outcome
Number & Rate of Loading Errors with
significant effect on M&B
Number & Rate of Loading Errors from an
incorrect loadsheet
Number & Rate of loadsheets accurately
produced
Proportion of load controllers trained on
aircraft types and their specific mass and
balance requirements
Usefulness
1. one criteria, no audience
2-4. some criteria, at least
one audience
5. majority of criteria, more
than one audience
Snapshot or
Contin.
Monitored
Cont.
Monitored
Cont.
Monitored
Data
Source
1
Outcome
2
Outcome
4
Activity
Varies
D2D
4
Activity
Snapshot
Trg Record
Availability
1. yes, easy
2. partially available, not
always reported
3. difficult to obtain
MOR
MOR
Accessibility (dependent
on coding review)
1. easy to export
3. detailed review
5. difficult to obtain
13
Risk Assessment - RAPT
 Risk Assessment
Prioritisation Tool
Tier 1 Safety
Risk Issue
Accident
Outcome
How important are
these controls?
How are these
controls degraded
by the risk issue?
How likely / what is
our exposure to
this risk issue
occurring?
What sector/s can
this risk issue be
present in?
What is the sector
contribution to the
overall risk picture?
What big ticket
accident outcomes
can this risk issue
result in?
How are these accident
outcomes distributed
globally?
14
Regulatory Oversight of Bowtie
 Industry
have bowties are part of their SMS

How were they created?
 How effective are they?
 How are they being used?
 Are they getting the most out of the models?
 How are they being updated?
 Where do they sit within the SMS?
15
Accident / Incident Investigation
 Which
scenario is relevant to the accident/ serious incident?
 Were
the expected controls present and how did they
perform?
 What
degraded the controls for the scenario to progress?
 Which
control prevented the scenario to progress into a more
catastrophic outcome?
16
Other Aviation Sectors

Airworthiness



Identified top risks on heat map
Deeper dive into the risks for maximise
improvement
General Aviation

Sector and outcome focussed



Hot Air Balloons, Flying Displays, EASA Aircraft etc
Highlight areas for oversight
Offshore Helicopters



Post Review Sept 2013 (CAP1145)
Top risks identified at SMS conference
Bowties being constructed with industry to
understand the risk further
 CAA & Industry use
17
Safety Assurance Review
 Understanding
the impact of change
 Highlighted concerns for internal independent audit
 Submitted recommendations
18
Pop-Ups
 Events,
risks, trends we are yet to fully understand and/or
want to manage more effectively:
RPAS – Remotely Piloted Aircraft System
 Volcanic Ash
 Birdstrikes
 Glider Conspicuity
 AGPLA – Attaching Ground Power to Live Aircraft
 State Safety Partnerships

19
Summary
 Significant
Seven Template is a solid foundation
 Established
Strategy
 Using
the templates to enhance risk management within
RSMS
 Appreciation
on control reliability and effectiveness to
enhance data collection and intelligence
 Variety
of uses for bowtie risk models
 Promote
our experience with industry and learn together
20