APPLYING HUMAN FACTORS ANALYSIS AND CLASSIFICATION SYSTEM
(HFACS) TO AVIATION INCIDENTS IN THE BRAZILIAN NAVY
by
Bruno Tadeu Villela
A Thesis Submitted to the college of Aviation Department of Applied Aviation Sciences
in Partial Fulfillment of the Requirements of the Degree of
Master of Science in Aeronautics
Embry-Riddle Aeronautical University
Daytona Beach, Florida
June 2011
APPLYING HUMAN FACTORS ANALYSIS AND CLASSIFICATION SYSTEM
(HFACS) TO AVIATION INCIDENTS IN THE BRAZILIAN NAVY
by
Bruno Tadeu Villela
This thesis was prepared under the direction of the candidate’s Thesis Committee Chair,
Dr. Guy M. Smith, Department Chair, Applied Aviation Sciences, Daytona Beach
Campus, and Thesis Committee Members Dr. Albert J. Boquet, Department
Chairman, Human Factors and Systems, Daytona Beach Campus, and
Daniel M. McCune, Associate Vice President for Safety, and
has been approved by the Thesis Committee. It was submitted to
the Department of Applied Aviation Sciences in partial
fulfillment of the requirements for the degree of
Master of Science in Aeronautics.
Thesis Review Committee:
____________________________________
Guy M. Smith, Ed.D.
Committee Chair
_________________________________
Albert J. Boquet, Ph.D.
Committee Member
_________________________________
Daniel M. McCune
Committee Member
________________________________
Marvin L. Smith, Ed.D.
Graduate Program Chair,
Applied Aviation Sciences
_______________________________
Guy M. Smith, Ed.D.
Department Chair,
Applied Aviation Sciences
________________________________
Tim Brady, Ph.D.
Dean, College of Aviation
_______________________________
Robert Oxley, Ph.D.
Associate Vice President of Academics
___________
Date
ii
Acknowledgements
First and foremost, I want to express my deepest and sincerest gratitude to my
family for their support throughout this 2-year project. To my beloved wife, Aline, who
helped me and endured with me the hardest times of our lives while in the U.S., I can
only say that without your love and understanding, this thesis would have never been
completed.
This thesis was made possible only by the support and encouragement from four
mentors:
Dr. MaryJo Smith – My MSA 605 professor who guided the development of my
mock thesis in my first term at ERAU;
Mr. Dan McCune – His enthusiasm for safety is contagious. His belief in a strong
safety culture and in the power of HFACS changed the way I see aviation safety;
Dr. Albert Boquet – His research experience applying HFACS provided the
expertise to develop my research design;
Dr. Guy Smith – His goal to finish my thesis in 14 weeks scared me, but his belief
and guidance made all the hard work worthy.
I wish to thank all of the professors and staff at ERAU who taught me and helped
me through the MSA program. All of you have been so patient in understanding the
limitations of my English language. Your continued support guaranteed all my success.
Special thanks to Jan Neal for editing my writing, and Dr. Weitzel and Roger Mason for
their friendship and support.
To my friends Orlando Olivas, Antony Ambrosino, Javier Perez-Albert,
Javier Alija, Marcela Vallejo, Nick and Amanda Kleoppel, Tammi Gibbs, and Angela
iii
Cox, thank for being my ERAU family for these last two years. I will never forget the
time we spent together either in the classroom or in the library or having fun during our
free time. I share this accomplishment with you all.
iv
Abstract
Researcher:
Bruno Tadeu Villela
Title:
Applying Human Factors Analysis and Classification System (HFACS) to
Aviation Incidents in the Brazilian Navy
Institution:
Embry-Riddle Aeronautical University
Degree:
Master of Science in Aeronautics
Year:
2011
The purpose of this thesis was to analyze Brazilian Naval Aviation incidents using the
Human Factor Analysis and Classification System (HFACS) to reveal underlying
conditions that contributed to human error. This study used the HFACS framework to
examine flight crew operations and maintenance operations. The source of the data was
141 incident summaries issued by the Brazilian Navy from 1999 to 2010 in which at least
one unsafe act was detected during the investigation. Maintenance-related unsafe acts
were identified in 60% of the cases. Of the 424 HFACS factors identified; skill-based
error, adverse mental state, decision error, and inadequate supervision were the most
prevalent, respectively. The researcher concluded safety initiatives like ORM, JAA, and
employing aviation psychologists are providing positive results within the Brazilian
Navy; however, the lack of formal human factors framework in investigations might be
hindering discovery of supervisory and organizational influences in the occurrence of
unsafe acts.
v
Table of Contents
Page
Acknowledgements .................................................................................................... iii
Abstract ....................................................................................................................... v
List of Tables............................................................................................................... x
List of Figures ............................................................................................................ xi
Chapter ................................................................................................................... 1
I
Introduction .......................................................................................... 1
Significance of the Study .......................................................... 3
Statement of the Problem .......................................................... 3
Purpose Statement..................................................................... 4
Delimitations ............................................................................ 4
Limitations and Assumptions .................................................... 4
Definitions of Terms ................................................................. 5
List of Acronyms ...................................................................... 6
II
Review of the Relevant Literature ......................................................... 8
Models of Accident Causation .................................................. 8
Sequence-of-Events Models .......................................... 9
Systemic Accident Models .......................................... 10
Epidemiological Models .............................................. 11
Reason’s Model of Accident Causation ............ 12
Human Error ........................................................................... 16
Theories of Error ......................................................... 16
vi
Human Factors Analysis and Classification System (HFACS) 19
Level 1 – Unsafe Acts ................................................. 20
Errors............................................................... 21
Violations ........................................................ 22
Level 2 – Preconditions for Unsafe Acts ...................... 22
Condition of Operators..................................... 23
Personnel Factors ............................................. 24
Environmental Factors ..................................... 24
Level 3 – Unsafe Supervision ...................................... 25
Inadequate Supervision .................................... 25
Planned Inappropriate Operations .................... 26
Failed to Correct Problem ................................ 26
Supervisory Violations ..................................... 26
Level 4 – Organizational Influence .............................. 26
Resource Management ..................................... 27
Organizational Climate .................................... 27
Organizational Process ..................................... 27
HFACS – Maintenance ................................................ 28
Aviation Accidents Investigation ............................................ 31
Human Factors Investigation ....................................... 34
Human Factors Investigation in Brazil ......................... 35
Accidents Investigation Reports .............................................. 36
Brazilian Navy Accidents Investigation and Reports .... 37
vii
Previous Research Applying HFACS ...................................... 38
Summary ................................................................................ 45
Research Questions ................................................................. 45
III
Methodology ...................................................................................... 46
Research Approach ................................................................. 46
Design and Procedures ................................................ 46
Population............................................................................... 49
Sources of the Data ................................................................. 49
Reliability ............................................................................... 51
Expert Reliability ........................................................ 51
Intra-rater Reliability ................................................... 52
Validity ................................................................................... 53
Treatment of the Data ............................................................. 53
Descriptive Statistics ................................................... 54
Reliability Testing ....................................................... 54
IV
Results ............................................................................................... 55
Descriptive Statistics ............................................................... 55
Incidents’ Demographic Variables ............................... 55
Chain of Factors Analysis Variables ............................ 59
Entire Framework Analysis Variables.......................... 63
Reliability Testing................................................................... 66
V
Discussion, Conclusions, and Recommendations ................................ 67
Discussion .............................................................................. 67
viii
Descriptive Statistics ................................................... 67
Incidents’ Demographic ................................... 68
Type of Incident ................................... 68
Operator and Model.............................. 68
Year ..................................................... 70
Location ............................................... 71
Flight Phase.......................................... 72
Cost...................................................... 73
Chain of Factors Analysis ................................ 74
Entire Framework Analysis .............................. 82
Reliability Testing ........................................... 88
Conclusions ............................................................................ 89
Recommendations ................................................................... 93
References ................................................................................................................. 95
A
Bibliography ..................................................................................... 100
B
Permission to Conduct Research ....................................................... 102
C
HFACS Framework .......................................................................... 104
D
Sample Data of Chain of Factors Analysis ........................................ 106
E
Sample Data of Entire Framework Analysis ...................................... 108
ix
List of Tables
Table
Page
1
Variables Collected from SIPAAerM Summaries ....................................... 50
2
Variables Used in the Chain of Factors Analysis ........................................ 50
3
Cohen’s Kappa Coefficient Interpretation ................................................... 53
4
Descriptive Statistics for Cost ..................................................................... 58
5
Chain of Factors Variables ......................................................................... 60
6
Most Common Chain of Factors ................................................................. 63
7
Entire Framework Analysis Variables......................................................... 64
8
Reliability Results ...................................................................................... 66
9
Brazilian Naval Aviation Fleet Composition ............................................... 69
x
List of Figures
Figure
Page
1
Domino Model ............................................................................................. 9
2
Swiss Cheese Model ................................................................................... 12
3
Basic Elements of a Productive System ...................................................... 14
4
Reason’s Model of Accident Causation ...................................................... 15
5
Summary of Psychological Varieties of Unsafe Acts .................................. 19
6
HFACS Framework.................................................................................... 20
7
HFACS-ME Framework............................................................................. 29
8
Description of Incident Type Variable ........................................................ 55
9
Description of Operator Variable ................................................................ 56
10
Description of Model Variable ................................................................... 56
11
Description of Year Variable ...................................................................... 57
12
Description of Location Variable ................................................................ 57
13
Description of Flight Phase Variable .......................................................... 58
14
Histogram of the Incidents’ Cost Variable, Entire Dataset .......................... 59
15
Histogram of the Incidents’ Cost Variable, Dataset without outlier ............. 59
16
Description of the Type Variable ................................................................ 60
17
Description of Level 1 Variable .................................................................. 61
18
Description of Level 2 Variable .................................................................. 61
19
Description of Level 3 Variable .................................................................. 62
20
Description of Level 4 Variable .................................................................. 62
21
Description of Entire Framework Analysis Variables ................................. 65
xi
1
Chapter I
Introduction
Since the late 1950's, the drive to reduce the accident rate has yielded
unprecedented levels of safety, so that today it is safer to fly in a commercial airliner than
to drive a car, or even walk across a busy New York City street (Shappell & Wiegmann,
2001). In the early years of aviation, aircraft were intrinsically unforgiving and
mechanically unsafe, leading the cause of accidents. However, in the modern era of
aviation, the literature indicated that between 70% and 80 % of aviation accidents were
attributed, at least in part, to human error (Shappell & Wiegmann, 2001). This contrast in
the genesis of accidents was directly related to the development of mechanical aircraft
technology and increasing aircraft reliability. Consequently, in the past two decades, a
growing number of aviation organizations began tasking their safety personnel with
developing safety programs to address the highly complex, and often nebulous, issue of
human error (Wiegmann & Shappell, 2001a).
The Human Factor Analysis and Classification System (HFACS) framework had
established a comprehensive, user-friendly tool for identifying and classifying the human
cause of aviation accidents (Wiegmann & Shappell, 2003). This tool has been used in the
military and civil environment in the United States of America and Canada, and its
reliability has been shown in many different studies conducted in the last five years
(Shappell & Wiegmann, 2003; Wiegmann & Shappell, 2003; Scarborough, Bailey, &
Pounds, 2005).
The Brazilian Navy Aeronautical Accidents Investigation and Prevention Service
(SIPAAerM) has been concerned with the role that human error plays in the Brazilian
Naval Aviation accidents and incidents. Since 1990, annually, the Aeronautical
2
Accidents Prevention Program (PPAA), published by SIPAAerM, has stated new
instructions and procedures to cover the gaps concerning human factors between the
regulation and new procedures or technology introduced in the aviation industry. In the
10 years since 1990, Crew Resource Management (CRM) and Operational Risk
Management (ORM) were introduced as orientation tools for pilots and maintenance
crew with the purpose of reducing human errors and violations (Marinha do Brasil,
1999).
In 2003, the PPAA focused on the use of CRM and ORM procedures to decrease
the probability of human error. It stated that error is a human characteristic; and any
procedure that reduces the necessity to evaluate, judge, or remember something could be
considered a safety improvement (Marinha do Brasil, 2003). However, the
organizational culture, its influences on the human error, and the cause of the human
error were not addressed.
In 2005, the Brazilian Navy published the second revision of its Aviation Safety
Manual, enhancing the regulation and procedures regarding human factors. This manual
established the present human factors classification for accidents and incidents reports,
introduced the Aviation Psychologist as a human factors specialist, and made it
mandatory for every operational squadron to have an Aviation Psychologist.
Additionally, the manual defined the difference between error and violation. The PPAA
of that year emphasized the new procedures and rules introduced by the new Aviation
Safety Manual (Marinha do Brasil, 2005a). Due to career requirements, some of these
Aviation Psychologists left their roles in operational squadrons, limiting the human
factors analysis of events for those organizations as well as the possibility of
3
implementing a human factors approach within those organizations (Marinha do Brasil,
2010).
Significance of the Study
The International Civil Aviation Organization (ICAO, 2006) stated in Annex 13
of the Chicago Convention, ―The sole objective of the investigation of an accident or
incident shall be the prevention of accidents and incidents‖ (p. 3-1). The actual human
factors approach used in accident investigation by the SIPAAerM has not used any
specific human error theoretical framework thus preventing accident and incident
investigators from discovering any underlying conditions that might have contributed to
human error in each specific case. This lack of a theoretical framework has made it
difficult to infer specific causes of human error, preventing the closure of the
investigative and prevention processes because the conclusions and recommendations
issued in an accident report or incident summary cannot be used to enhance prevention
approaches within Brazilian Naval Aviation.
Statement of the Problem
Accidents and incidents have been analyzed using the HFACS framework in
order to reveal the underlying human causes of these mishaps in many different countries
and in different types of operations. To date, the SIPAAerM has not yet used HFACS to
analyze its aviation safety data. The actual human factors approach used in accident and
incident investigations by SIPAAerM has followed the standards defined in the
aeronautical regulations established by the Aeronautical Accidents Prevention and
Investigation Center (CENIPA) from the Brazilian Air Force. However, this approach
has not sought to detect the conditions that lead to human error, an omission that could
hinder interventions that could be employed to prevent future recurrences of the
4
accidents, incidents, and undesirable events. This study used the HFACS and the
HFACS-MEDA taxonomies to analyze the Brazilian Naval Aviation incidents reports in
order to discover underlying conditions that might have contributed to human error to
those cases.
Purpose Statement
The purpose of this study was to assess the HFCAS applicability in the Brazilian
Naval Aviation incidents in order to discover any underlying conditions that contributed
to human error.
Delimitations
The exclusive focus of this study was the analysis of the Brazilian Naval Aviation
Incidents and Ground Occurrences Summaries approved and issued by the SIPAAerM
from 1997 to 2010. The analyzed summaries were neither modified nor criticized. The
investigations that lead to the conclusions and recommendations contained on the
summaries were not subject to analysis, and the actual investigative documents were not
read by the researcher.
Limitations and Assumptions
Due to time and money constraints, only the researcher, who is trained in the
HFACS framework, performed the analysis of the data. Additionally, the fact that all of
the summaries were written in Portuguese prevented the researcher from using Englishspeaking volunteers as additional coders. The researcher developed the instrument to
collect the data, since the differences in the incident summaries from the previous
application of HFACS precluded him from using the same instruments. The assumption
was made that all investigations that lead to the publication of the analyzed summaries
5
were done applying the human factors approach approved by the SIPAAerM at the time
of the incidents.
Definitions of Terms
Aeronautical Accidents Prevention Program: Brazilian Navy document that
contains the planning for the activities related to aviation safety that will be developed in
a determined period of time and responsibility area, in order to prevent aeronautical
accidents or to minimize their consequences (Marinha do Brasil, 2005a).
Aeronautical Occurrence: Term used as a general reference to an aeronautical
accident, severe incident, incident, or ground occurrence (Marinha do Brasil, 2005a).
Aeronautical Incident: Every occurrence associated with the operation of an
aircraft in which there was an intention of flight but cannot be characterized as an
accident, and it affected or could have affected the safety of the operations (Marinha do
Brasil, 2005a).
Contributory Factor: Condition (act, fact, omission, or combination) that, along
with others, in sequence or as a consequence, results in an aeronautical occurrence or
contributes to the severity of its consequences (Marinha do Brasil, 2005a).
Grave Incident: Incident that happened under such circumstances that an accident
almost occurred. The difference between a grave incident and an accident is
characterized by the severity of the consequences (Marinha do Brasil, 2005a).
Ground Occurrence: Every occurrence involving an aircraft in which there was
no intention of flight but which results in damage or injury (Marinha do Brasil, 2005a).
Violation: Intentional disregard to a formally established rule or procedure
(Marinha do Brasil, 2005a).
6
List of Acronyms
ANAC
Agência Nacional da Aviação Civil [Brazilian National Civil
Aviation Agency]
ATC
Air Traffic Control
ATSB
Australian Transport Safety Bureau
CENIPA
Centro de Investigação e Prevenção de Acidentes Aeronáuticos
[Brazilian Air Force Aeronautical Accidents Prevention and
Investigation Center ]
CFIT
Control Flight into Terrain
CRM
Crew Resource Management
FAA
Federal Aviation Administration
GA
General Aviation
HFACS
Human Factor Analysis and Classification System
HFACS-ME Human Factor Analysis and Classification System – Maintenance
Extension
ICAO
International Civil Aviation Organization
JAA
Jornada de Atividade Aérea [Aeronautical Activity Journey]
JSAT
Joint Safety Analysis Team
JSIT
Joint Safety Implementation Team
MEDA
Maintenance Error Decision Aid
MEIMS
Maintenance Error Information Management System
MRM
Maintenance Resource Management
NTSB
National Transportation Safety Board
ORM
Operational Risk Management
7
OSAv
Oficial de Segurança de Aviação [Aviation Safety Officer]
PPAA
Programa de Prevenção de Acidentes Aeronáuticos [Aeronautical
Accidents Prevention Program]
SIPAAerM
Serviço de Investigação e Prevenção de Acidentes Aeronáuticos da
Marinha [Brazilian Navy Aeronautical Accidents Investigation and
Prevention Service]
SHEL
Software, Hardware, Environment, and Liveware
SPSS
Statistical Package for the Social Sciences
8
Chapter II
Review of the Relevant Literature
The motivation for conducting accident and incident investigations extends
beyond discovering the cause of any event. The focus has to be on the lessons learned
with a view to the prevention of similar events. The greater the volume of information
that can be gathered, the more complete is the picture that can be gained and the firmer
the basis of any recommendations for future improvements. The role of human errors in
accident and incident investigation has received increased attention in recent years due to
increased aircraft reliability introduced by technological advancements (Shappell &
Wiegmann, 2000). The inclusion of human factors as a potential issue in accident and
incident investigation has come late on the investigative scene. However, undervaluing
or underestimating the human factors aspect of these events had probably led to an
unbalanced and incomplete picture in attempting to determine what and why an accident
happened (Baker, 1999). Likewise, the use of improper tools or procedures might have
the same hindering effect in achieving the appropriate recommendations and preventive
approach to each type of event.
Models of Accident Causation
In complex systems, there is no single cause to an accident or incident. In fact,
only a large number of contributory factors are necessary and only jointly sufficient to
overwhelm the systems’ defenses and cause an accident (Dekker, 2006). Accident
models are necessary because they facilitate the communication and understanding and
help develop a single point of view to the mishap. The model determines the way a
person views an accident, and, in particular, how a person views the role of humans
(Hollnagel, 2004). Moreover, an accident model helps the investigator determine what to
9
look for in an accident investigation, bringing relative order to the rubble of failures that
were found; it is a framework that an accident investigator uses to explain the relationship
among the facts found in the course of an investigation.
Sequence-of-events models. The simplest type of accident model describes the
accident as a chain of events that leads up to a failure (Dekker, 2006). The most famous
model in this type is known as the Domino model, in that one event trips the next, like
dominos falling. This model focuses on what went wrong, leaving out additional
information that might have played a relevant role in the accident. However, an analysis
is not limited to just one sequence of events, because many chains of events might have
contributed to the final mishap (Hollnagel, 2004). Figure 1 presents a common
illustration of the Domino model.
Figure 1. Domino model. Note. Adapted from Hollnagel, 2004.
These models are attractive because they encourage thinking in a causal sequence
instead of causal nets. It is much easier to follow a linear reasoning than a parallel
reasoning. Also, linear reasoning is easier to graphically represent, facilitating the
communication of the results. If causes and effects were obvious, and if there were not
10
too many causes that led to the effects, these models could work well for the events
immediately before a mishap (Dekker, 2006).
Many researchers use Fault Tree Analysis or Root Cause Analysis as methods to
analyze complex accidents in which many chains of events occurred simultaneously. The
weakness of these models is the symmetry of cause-effect relationships. In the real
world, these relationships are not linear and are much more complex than these models
are able to represent and support (Hollnagel, 2004).
Systemic accident models. These models see accidents as emerging from
interactions between system components and processes instead of failures within them.
By this view, accidents occur during the normal operation of the system; they are a byproduct of people and organizations trying to pursue success with imperfect knowledge
and resource constraints. The focus in these models is the overall, not the parts. Human
error or equipment failures are meaningless if the social-technical system that shaped the
human’s performance and the equipment design is not taken into account (Dekker, 2006).
The advantage of systemic models is their emphasis that accident analysis must be
based on an understanding of the functional characteristics of the system, rather than on
assumptions or hypotheses. The models advocate a search for unusual dependencies and
common conditions that, from experience, are associated with accidents. Moreover,
systemic models can deal with non-linear relationships that the sequential event models
cannot. Systemic models have their roots in control theory and chaos theory,
emphasizing its application complexity. This complexity leads to problems in
representing accidents graphically, making communication and understanding more
difficult (Hollnagel, 2004).
11
Epidemiological models. These models describe accidents as the outcome of a
combination of factors, some latent and some active, that happened together in space and
time. Accidents, by these models, are related to latent failures that hide in every part of
the system, from management decisions to procedures to equipment design. These
models empower the possibility of seeing more complex connections among various
factors, not only the causal series (Dekker, 2006).
All of these models were developed based upon four main features. The first
feature is performance deviation, which is a generic term for unsafe acts or human error.
However, in these models, performance deviation can also be applied to equipment or
any other technological component. The second feature is environmental conditions that
can lead to performance deviations. Environmental conditions exist to both humans and
equipment, but affect them in different ways that must be analyzed. The third feature is
barriers, representing all obstacles that could prevent the unexpected consequences from
occurring. In this sense, the barriers could stop the development of the accident at the
last moment. The last feature is latent conditions, which are present within the system
even prior the onset of an accident sequence. Latent conditions might exist in different
forms that combine with active failures to generate a mishap. This last feature might
have numerous causes, such as managerial decisions, design failures, or maintenance
failures (Hollnagel, 2004).
Despite the linearity and simplicity that characterize these models, they have been
helpful in conceptualizing the resulting imperfect organizational structure that led to a
mishap. The well-known Swiss Cheese analogy, first introduced by Reason (1997), is the
classical epidemiological model, as represented in Figure 2, in which the aforementioned
characteristics can be easily visualized. In Figure 2, it is simple to imagine the
12
anteceding organization factors as a series of porous defense layers. However, the
weakness of these models is that they do not allow the analysis of the processes that
created the hole in the layers of defense. A deeper and separate analysis of each layer is
necessary to uncover the cause of this gap (Dekker, 2006).
Figure 2. Swiss cheese model. Note. Adapted from Wiegmann and Shappell, 2003.
Reason’s model of accident causation. The most recognized aviation accident
model was presented by Reason (1997). Using this model, Reason defined organizational
accidents as having multiple causes that involve numerous people operating at different
levels of the organization. Although rare, these events have led to catastrophic
consequences within complex modern technology industries, such as nuclear power
plants, aviation, and chemical processing plants (Reason).
According to Reason’s (1997) approach, fundamental elements of an organization
must work together to achieve some goal, which can be a service or a product. These
elements taken together are called a productive system. In the aviation industry, the goal
is to conduct safe flight operations, no matter what type of operation an organization is
involved. Pilots and maintenance personnel are the line operators that conducted the final
13
activities essential to achieve the organizational goal. During productive activities, line
operators have to interact with equipment, rules, procedures, and other people within the
system (Shappell & Wiegmann, 2003).
However, before productive activities can occur, certain conditions need to be
present. Without reliable equipment and well-trained professionals, the organization
would never achieve its goal. Furthermore, management and supervision personnel are
needed across numerous departments within the organization, such as operations,
maintenance, and training. These managers also need guidance, as well as the personnel
and money to perform their duties efficiently. This support comes from decision-makers
who are even further up the organizational hierarchy, and are responsible for setting goals
and managing available resources. These are the individuals who have the job of
balancing the competing goals of productivity and safety (Shappell & Wiegmann, 2003).
Too much emphasis on the production aspect can lead to catastrophic outcomes, whereas
too much emphasis on safety can lead the organization to bankruptcy or discontinuing
operations (Reason, 1997).
Moreover, executive decisions are typically based on social, economic, and
political inputs coming from outside the organization, as well as feedback provided by
managers and workers from within (Shappell & Wiegmann, 2003). Executives also
develop, consciously or not, defenses within the organization in order to prevent
breakdowns; thus, creating layers of protection. Engineered safety features, safety
procedures, and barriers are examples of the numerous defenses that can be employed
(Reason, 1997). Figure 3 depicts the basic elements of a productive system within an
organization.
14
Figure 3. Basic elements of a productive system. Note. Adapted from Reason, 1990.
Accidents occur when there are breakdowns in the interactions among the
components involved in the production process, degrading the integrity of the system and
making it more vulnerable to operational hazards. These failures can be depicted as a
hole within the different layers of the system, transforming a productive process into a
failed or broken down one (Shappell & Wiegmann, 2003). Since humans design,
operate, and manage the system, it is not hard to understand that human actions and
decisions are involved in all accidents. This human contribution can occur in two
different ways. Active failures, even error or violations, are committed by the system
operators and are likely to have direct impact on the safety of the system. Latent
conditions arise from strategic and other top-level decisions and combine with local
circumstances and active failures to penetrate the system’s defense. They may be in
place for years without being detected and can increase the likelihood of active failures
throughout the organization by creating local factors promoting unsafe acts (Reason,
1997).
15
Within the concept of latent failures, Reason (1997) has defined three more levels
of human failure. The first involves the condition of the operator as it affects
performance, and is termed Preconditions for Unsafe Acts. This level involves
conditions such as mental fatigue and poor communication and coordination practices.
The next level, Unsafe Supervision, directly influences the previous level in the sense that
interventions and strategies to prevent Preconditions for Unsafe Acts lies higher in the
system. Sometimes operators are exposed to situations that are not under their control.
The supervisory personnel have responsibility over this level, but management personnel
are also responsible for this level. The organization itself can affect performance at all
levels, unveiling the last level of human failure, Organizational Influences (Shappell &
Wiegmann, 2000). Figure 4 presents the Reason’s model of accident causation.
Figure 4. Reason’s model of accident causation. Note. Adapted from Reason, 1990.
Reason's (1990) model of accident causation revolutionized the aviation industry
by integrating human error into an epidemiological accident model. That is the reason
why it is one of the most accepted models in the aviation industry. Nevertheless, it is
16
simply a theory with few details on how to apply it in the real world (Shappel &
Wiegmann, 2000). In fact, the model has two limitations: (a) it fails to identify the exact
nature of the holes in the cheese; and (b) it is primarily descriptive, not analytical
(Shappell & Wiegmann, 2003).
Human Error
Most human errors are not relevant and are quickly forgotten or forgiven. The
relative minor effect that results from errors justifies the relative inattention everyone
pays to them. In many cases, errors can also play the role of a learning experience,
mainly to children or people who are not yet familiar with the task they are performing.
Training professionals recognized the value of errors in the learning environment a long
time ago, and have developed simulators that enable trainees to experience errors without
the real-world consequences (Strauch, 2002). This is vital in aviation training because
the above-ground environment is not permissive to human error due to the high
consequences of accidents.
The common and trendy claim that human error is associated with something
between 70 to 80% of the aviation accidents, already cited in this study, does little help in
explaining how and why accidents happened. One can conclude that humans are the
hazardous part of a system. In addition, the term, human error, conveys the impression
that all unsafe acts can be stacked into the same category, which is not true (Reason,
1997). The aviation safety professional has to understand that human error is the
symptom of deeper trouble in the system; it is systematically correlated to features of
people’s tools, tasks, and operating environments (Dekker, 2006).
Theories of error. The modern error theory that correlates error with the
surrounding environment and system was not shared by past theorist. Freud, as cited by
17
Strauch (2002), believed that error was a product of the unconscious traits of the person.
People who performed errors were considered less effective and probably more deficient
than those who did not. Recent studies discovered serious methodological deficiencies in
the studies that were the basis for that theory, weakening the present adoption of it, and
stimulating further research (Strauch).
Norman (1988) evaluated the cognitive and motor aspects of error to establish the
difference between two types of error, slips and mistakes. Whereas slips are errors in the
formation of intentional acts or the faulty triggering of schemas, mistakes are errors of
thought in which a person’s cognitive activities lead to actions or decisions that are
contrary to what was intended. Norman argued that the application of the lessons learned
in slips would help to design new schemas and therefore reduce the likelihood of slips, as
cited in Strauch, 2002.
As cited in Strauch, 2002, Rasmussen expanded the cognitive aspect of human
error described by Norman, but with remarkable differences (Strauch, 2002; Reason,
1990). Whereas Norman was primarily concerned with the usually inconsequential
actions of slips that occur in the normal course of daily life, Rasmussen’s theory is
directed at the more serious errors made by those in supervisory control of industrial
installations (Reason, 1990). Rasmussen defined three types of operator performance
levels and three associated errors: (a) skill-based performance, (b) rule-basedperformance, and (c) knowledge-based performance. Skill-based performance errors are
analogous to Norman’s slips, largely errors of execution (Strauch, 2002). Rule-based
performance errors result from the misclassification of situations leading to the
application of the wrong rule or from the incorrect recall of procedures (Reason, 1990).
Knowledge-based performance errors result from shortcomings in the operator
18
knowledge or limitations in the ability to apply existing knowledge to new situations
(Strauch, 2002).
In 1990, Reason published the treaty that is today the most accepted and the most
applied human error theory within the aviation industry. He enlarged the focus of the
earlier definitions of errors and established further distinctions among the three basic
types of errors, as cited in Strauch, 2002. Along with slips and mistakes, Reason defined
lapse, which is characterized as a primarily memory error, as the third type of error.
Moreover, he used Rasmussen levels of performance to categorize these error types.
Slips and lapses were associated with skill-based performance level, whereas mistakes
were associated with rule-based or knowledge-based level (Reason, 1990).
Another relevant contribution from Reason (1990) was the distinction between
errors and violations. The aforementioned error classification was restricted to individual
information processing, and as such offered only a partial account of the possible
varieties of abnormal behavior. The missing part is the regulated social environment in
which the human behavior occurs. Violations can only be defined concerning the social
context in which behavior is governed by operating procedures, codes of practices, and
rules. Hence, violations were defined as deliberate deviations from those practices
deemed necessary to accomplish safe operations within the system (Reason, 1990). A
summary of the psychological varieties of unsafe acts, classified initially according to
whether the act was intended or unintended and then distinguishing errors from violations
is presented in Figure 5.
19
Figure 5. Summary of psychological varieties of unsafe acts. Note. Adapted from
Reason, 1990.
Human Factors Analysis and Classification System (HFACS)
The intention to provide a suitable foundation for conducting a comprehensive
analysis of human error led many researchers to develop different frameworks to address
human errors. These frameworks were based in six major human error perspectives:
(a) cognitive, (b) ergonomic, (c) behavioral, (d) aeromedical, (e) psychological, and
(f) organizational. Each framework and its perspective approach have different strengths
and weaknesses. Wiegmann and Shappell (2003) stated that while a few frameworks
have enjoyed limited success, none has come to the almost universal acceptance and
recognition that Reason’s (1990) model of accident causation has received.
The HFACS framework was originally developed by Shappell and Wiegmann
(2003) for the United States Navy and Marine Corps as an accident investigation and data
analysis tool. Since its development, other organizations, such as the Federal Aviation
Administration (FAA), have explored the use of HFACS as a complement to pre-existing
systems within civil aviation, in an attempt to capitalize on gains made by the military
(Shappell & Wiegmann).
20
The HFACS was established to bridge the gap between theory and practice by
providing investigators with a comprehensive, user-friendly tool for identifying and
classifying the human cause of aviation accidents (Wiegmann & Shappell, 2003). The
entire framework includes 19 factors within four levels of human failure: (a) unsafe acts,
(b) preconditions for unsafe acts, (c) unsafe supervision, and (d) organizational
influences. Figure 6 presents the HFACS framework. The unsafe acts and the
preconditions for unsafe acts levels have also HFACS categories before reaching the
HFACS factor level.
Figure 6. HFACS framework. Note. Adapted from Wiegmann & Shappell, 2003.
Level 1 – unsafe acts. Following the same definitions established by Reason
(1990), the first level of the HFACS framework can be divided into two categories: (a)
errors and (b) violations. Errors represent the mental or physical activities of the
21
operators that fail to achieve the intended outcome (Wiegmann & Shappell, 2003).
Because humans are remarkably prone to make errors, this is the most common unsafe
act detected in accidents investigations. On the other hand, violations, which can be
defined as willful disregard to rules and regulations related to flight safety, are not very
often encountered (Wiegmann & Shappell, 2003). Both errors and violations can be
further categorized into HFACS factors.
Errors. This category of unsafe acts can be divided into three different HFACS
factors. The first HFASC factor is skill-based error. In aviation, this type of error
encompasses the failures that are widely known as ―stick-and-rudder‖ or any basic skill
that a pilot uses without conscious thought. In this type, attention or memory failures
have been linked to some skill-based error like breakdown in instrument scan patterns,
task fixation, or omitting items in a checklist. Flying technique errors reflect each
individual’s innate ability and aptitude (Shappel & Wiegmann, 2000; USNSC, 2001;
Wiegmann & Shappell, 2003).
The second HFACS factor under the error category is decision error. This type of
error is characterized by the intentional execution of a plan that turned out to be
inadequate or inappropriate for that specific situation. Often called ―honest mistakes‖,
these errors are the most vastly investigated. The highly structured tasks within the
aviation environment force pilots to use procedural decision-making. There are
procedures for every single part of the flight and the misapplication of any of them
represents a classic example of this type of error. Simultaneously, there are numerous
situations in which there are no procedures established and a decision has to be made. In
this case, a choice has to be made among all the available options (Shappel & Wiegmann,
2000; USNSC, 2001; Wiegmann & Shappell, 2003).
22
The third HFACS factor in this category is perceptual error. Reasonably, the
perception of the world around an individual influences the way that person reacts to any
situation. When the person’s perception does not reflect the actual world, an error often
occurs. Visual illusion, spatial disorientation, and the misjudgment of attitude, altitude,
or airspeed might be caused when sensory inputs are degraded or unusual, and are the
most common errors in this category (Shappel & Wiegmann, 2000; USNSC, 2001;
Wiegmann & Shappell, 2003).
Violations. This category of unsafe acts is divided into two HFACS factors. The
first HAFSC factor is routine violation. This form of violation tends to be customary by
nature, often tolerated by superior levels, and, unfortunately, in some cases, sanctioned
by supervisors. The mishandling of the violation can lead to the idea that the specific
behavior is approved, despite the clearly contrary to the regulations or procedures in
place (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann & Shappell, 2003).
The other HFACS factor under the violation category is exceptional violation.
This form of violation occurs when a person deviates from the individual’s typical
behavior pattern or from the behavior accepted by the supervisory authority, usually
appearing as isolated events. The problem presented to organizations is that exceptional
violations are particularly hard to predict and prevent due to the lack of any indication
that they might happen (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann &
Shappell, 2003).
Level 2 – preconditions for unsafe acts. Focusing only on the unsafe acts is not
enough to unveil the failures within the organization. Nonetheless, this is still the most
common approach to human error in many accident investigations. The analysis of the
preconditions for unsafe acts, which include the condition of the operator, environmental,
23
and personnel factors, might help in understanding the reasons why the unsafe acts
occurred (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann & Shappell, 2003).
Condition of operators. It is clear that the conditions of an individual directly
influence performance in whatever task is accomplished. The first HFACS factor under
this category of preconditions for unsafe act is adverse mental states. Mental preparation
is an essential part of proper performance. Task fixation, distraction, and mental fatigue
not only affect performance but also increase the likelihood that an error will occur. At
the same time, overconfidence or hazardous attitudes increase the likelihood that a
violation will occur. This factor accounts for these mental states in the chain of events
(Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann & Shappell, 2003).
The second HFACS factor in this category is adverse physiological states, and
encompasses medical or physiological conditions that prevent safe operations. The
emphasis in this factor lies on the innumerable pharmacological and medical
abnormalities that might influence performance, but that are often overlooked in the
aviation industry. Due to the particular environment in which flight is conducted,
pathologies that are not significant on the ground can disrupt an operator’s ability to
perform in flight. Also, some pathologies make the operator more prone to spatial
disorientation and visual illusions (Shappel & Wiegmann, 2000; USNSC, 2001;
Wiegmann & Shappell, 2003).
The last HFACS factor in this category is physical/mental limitations, and refers
to the situations in which the capabilities of the individual are overwhelmed by
operational requirements. This factor accounts for not only the basic sensory and
information processing limitations, but also for individuals who are not physically or
mentally compatible with the aviation environment requirements. These last issues are
24
often not addressed due to political reasons, which increase the likelihood that they might
be present in mishaps (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann &
Shappell, 2003).
Personnel factors. This category of precondition for unsafe acts accounts for the
things that an operator can do to create these preconditions. This category is further
divided into two HFACS factors. The first, Crew Resource Management (CRM), is a
cornerstone in the aviation history, which makes its understanding much easier. This
factor refers to the occurrences of poor coordination or poor communication among
personnel involved in the operation, encompassing flight crew, air traffic control,
maintenance, and support crew (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann
& Shappell, 2003).
The other HFACS factor under this category is personal readiness representing
the individual’s failure to prepare mentally and physically to appropriately perform at
high levels. Violations of crew rest requirements and self-medication are classic
examples of this factor. It is relevant to explain that the aforementioned violations of
rules differ from the unsafe act in time, space, and consequence. They influence an
individual’s fitness for the flight, but do not directly cause any mishap (Shappel &
Wiegmann, 2000; USNSC, 2001; Wiegmann & Shappell, 2003).
Environmental factors. This precondition for unsafe acts category refers to
environmental factors that can contribute to unsafe acts. It is divided into two HFACS
factors. The first HAFCS factor is represented by the physical environment. Being one
of the most vastly documented influences in flight crew performance, the physical
environment can impose numerous limitations to performance. It includes both the
ambient environment and operational environment. For example, unsafe acts can be a
25
result of reduction in pilot’s concentration due to dehydration while flying in a hot
environment (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann & Shappell,
2003).
The other HFACS factor under this category is technological environment. This
factor, recently introduced into the aviation due to the technological revolution of the last
decade, is not as well documented as the previous factor. This factor accounts for the
design of equipment and controls, automation and display characteristics that can
influence an operator’s performance. The introduction of automation in the cockpit and
its interaction with pilots has revealed nuances that were not anticipated. Confusion can
arise from many situations involving recently introduced equipment; thereby increasing
the likelihood that an unsafe act might occur (Shappel & Wiegmann, 2000; USNSC,
2001; Wiegmann & Shappell, 2003).
Level 3 – unsafe supervision. Recalling Reason’s (1990) accident model, unsafe
supervision is considered as a latent condition that affects an operator’s performance by
directly disturbing the operator or disturbing the environment in which the operator
performs. It is divided into four HFACS factors: a) inadequate supervision, b) planned
inappropriate operations, c) failure to correct problems, and d) supervisory violations
(Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann & Shappell, 2003).
Inadequate supervision. The supervisor’s responsibility is to provide adequate
condition for the job to be done safely and efficiently by offering proper guidance,
training leadership, and oversight. This responsibility comes along with accountability
for the operator’s acts. The lack or inappropriateness of guidance and oversight sets the
stage for operator’s unsafe acts (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann
& Shappell, 2003).
26
Planned inappropriate operations. This HFACS factor refers to the situations in
which operators are put in a position where they face unacceptable risk to their safety. In
these circumstances, performance is adversely affected. Improper scheduling and crew
pairing and failure to afford sufficient crew rest might also set the stage for an operator’s
unsafe acts (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann & Shappell, 2003).
Failed to correct problem. Within the supervisor’s responsibility lies the
enforcement of rules and procedures through correcting inappropriate behavior or
operations. This HFACS factor accounts for the instances when these behaviors or
operations related to safety areas are known by supervisors but are not rectified. The
failure to correct a problem certainly fosters an unsafe atmosphere, directly influencing
the occurrence of unsafe acts (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann &
Shappell, 2003).
Supervisory violations. This HFACS factor refers to the supervisor’s willful
disregard of existing rules and regulations. Permitting a pilot without the proper
qualification to operate the aircraft and failure to keep adequate records are examples of
these supervisory violations. Despite its rarity, this kind of violation invariably sets the
stage for catastrophic mishaps (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann
& Shappell, 2003).
Level 4 – organizational influence. Fallible decisions of the top level
management of the organization can result in latent defects, directly affecting supervisory
practices as well as conditions and actions of the operators. Often, these latent failures
are difficult to be detected by safety professionals because they are also within the
boundaries of the organization. This level is divided into three HFACS factors:
27
(a) resource management, (b) organizational climate, and (c) organizational process
(Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann & Shappell, 2003).
Resource management. This HFACS factor relates to the overall organizational
resources encompassing human resources, monetary assets, equipment, and facilities. As
explained by Reason’s (1990) accident model, any organization faces conflicting interests
that pit production goals against protection goals. For example, excessive cost-cutting, a
lack of funding for proper and safe equipment might adversely affect an operator’s
performance and safety. This factor accounts for all organizational decisions that
influence operations and safety (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann
& Shappell, 2003).
Organizational climate. Encompassing structure, policies, and culture, this
HFACS factor can be defined as the prevailing atmosphere or environment within the
organization. Ill structured organizations have a greater probability for accidents.
Unclear and inconsistent policies might lead to degraded safety levels. Finally, an
organization culture is defined by the unofficial rules, values, and attitudes adopted
within the organization. When the culture develops against the formal rules and
regulations, it might set the stage for reduction of safety levels (Shappel & Wiegmann,
2000; USNSC, 2001; Wiegmann & Shappell, 2003).
Organizational process. The last HFACS factor describes the organizational
decisions and rules that regulate the everyday activities within an organization, including
the establishment and use of standard operating procedures and formal methods of
oversight. The work conditions set up by management, such as operational tempo and
production quotas, might be detrimental to safety. Also, to ensure that the working
environment is safe and productive, management should observe and check resources and
28
climate, playing the appropriate oversight role. Otherwise, safety might be again
jeopardized (Shappel & Wiegmann, 2000; USNSC, 2001; Wiegmann & Shappell, 2003).
HFACS – Maintenance. The HFACS framework was developed for classifying
accidents and incidents associated with aircrew errors (FAA, 2008). The general
definition of the HFACS factors made use of numerous examples of flight crew errors
and the associated factors that lead to accidents or incidents (Wiegmann & Shappel,
2003). Maintenance operations within the aviation industry were not included in the
framework development. Besides being one of the most relevant parts of the aviation
industry, aircraft mechanics face challenges similar to pilots while accomplishing their
tasks.
The U.S. Navy was the first organization to detect the opportunity to modify the
HFACS framework in order to cover the existing gap in the human factor analysis of its
maintenance operations. In 1998, Schmorrow conducted a research at the Naval
Postgraduate School in which a HFACS Maintenance Extension (HFACS-ME)
framework was designed specifically for analyzing aviation maintenance-related
mishaps. However, some relevant changes were made within the conceptual background
of the framework. The unique level that did not change was the unsafe acts level. The
second level was renamed to working conditions, the third level to maintainer condition,
and the fourth level to supervisory conditions (Schmorrow, 1998; USNSC, 2001). Figure
7 presents the HFACS-ME framework. The aforementioned changes also influenced the
way the framework is presented.
29
Figure 7. HFACS-ME framework. Note. Adapted from Schmorrow, 1998.
While successfully applied to the naval aviation accidents at that time, these
changes excluded the organizational influence on the accidents’ chain of events
(Schmorrow, 1998). Additionally, changing not only the names but also the focus of the
three HFACS levels prevented this framework from being used together with the primary
HFACS framework in databases where accidents and incidents related to aircrew error or
maintainer error are not separated.
Despite these problems, the HFACS-ME has been applied in other research. In
2004, Krulak analyzed the Maintenance Error Information Management System
30
(MEIMS) database using the HFACS-ME framework. The database was a joint effort of
NASA, the FAA, and the U.S. Navy to promote analysis of aircraft accidents and
incidents. The research analyzed 1,016 civilian aviation mishaps that occurred between
1996 and 2000. The results showed that the framework was reliably applied and that
inadequate supervision was the most common HFACS-ME factor identified (Krulak).
In 2010, Rashid, Place, and Braithwaite analyzed 58 helicopter maintenancerelated safety occurrences in order to study helicopter mishaps survivability and the
severity distribution of the occurrences. The research encompassed helicopter accidents
and incidents that occurred in Australia, Canada, New Zealand, UK, and USA, from 1995
to 2005, which were exclusively related to maintenance issues. The results were
categorized into the four levels of the HFACS-ME framework, comparing the factors
within each level. More remarkably, the researchers concluded that many causal factors
of the mishaps were deeply rooted into the organizational and managerial levels (Rashid
et al.). The HFACS-ME did not support a further analysis to include the organizational
level.
At the time this last research was conducted, the FAA had already released and
published the Human Factors Guide for Aviation Maintenance and Inspection, which
introduced a new maintenance version of the HFACS framework (FAA, 2008). The
intention of this new version was to combine the HFACS with the Maintenance Error
Decision Aid (MEDA) methods to classify maintenance errors in the aviation industry.
MEDA, a joint collaboration between Boeing representatives from United Airlines,
British Airways, Continental Airlines, the International Association of Machinists, and
the FAA, started out as an error investigative process (Boeing, n.d.).
31
In the early 2000’s, MEDA changed the philosophy to aggregate the violation
concepts and became an event investigation process. MEDA philosophy is based on the
same premise as the HFACS that superior levels of an organization directly affect the
performance of the inferior levels. The bottom level is represented by the maintainer, the
intermediate by supervisors, and the top level by management. The work environment
around the maintainers also influences their performance, and its level is placed between
the supervisor and the maintainer level (FAA, 2008).
With so many characteristics in common, the combination of the HFACS and
MEDA generated a framework, the HFACS-MEDA, which follows the same design and
outline as the primary HFACS framework. The levels and the factors are both named the
same way, yielding the same definitions. The most relevant difference lay in the
individual performing an unsafe act. Whereas the flight crew was the operator within the
HFACS, the maintainer was the operator within the HFACS-MEDA. The environment in
which the operators perform also differs, since a flight deck does not mirror the same
characteristics as a maintenance repair station. However, these differences did not
prevent the use of the same concepts and definitions (FAA, 2008). Hence, HFACS and
HFACS-MEDA yield results that are comparable against and within each other.
Aviation Accidents Investigation
In Annex 13, 9th edition, to the Convention on International Civil Aviation ICAO
(2006) defines an accident as:
An occurrence associated with the operation of an aircraft which takes place
between the time any person boards the aircraft with the intention of flight until
such time as all such persons have disembarked, in which:
a) a person is fatally or seriously injured as a result of:
32
— being in the aircraft, or — direct contact with any part of the aircraft, including
parts which have become detached from the aircraft, or
— direct exposure to jet blast, except when the injuries are from natural causes,
self-inflicted or inflicted by other persons, or when the injuries are to stowaways
hiding outside the areas normally available to the passengers and crew; or
b) the aircraft sustains damage or structural failure which:
— adversely affects the structural strength, performance or flight characteristics
of the aircraft, and
— would normally require major repair or replacement of the affected
component, except for engine failure or damage, when the damage is limited to
the engine, its cowlings or accessories; or for damage limited to propellers, wing
tips, antennas, tires, brakes, fairings, small dents or puncture holes in the aircraft
skin; or
c) the aircraft is missing or is completely inaccessible (p.1-1).
Additionally, ICAO states in the same document that an incident is ―an occurrence, other
than an accident, associated with the operation of an aircraft which affects or could affect
the safety of operation‖ (ICAO, 2006, p.1-1). Contracting States to the Convention do
not have to adopt the same definition. However, any differences between national
regulations and practices and the International Standards contained in Annex 13 should
be notified to the ICAO.
In Brazil, the CENIPA adopted both definitions, but added two other types of
aviation occurrences: grave incident and ground occurrence. The first occurrence is
defined as ―an incident that happened under such circumstances that an accident almost
happened‖ (Comando da Aeronáutica, 2008a, p.25). The difference between a grave
33
incident and an accident exist only on the consequences. The second occurrence can be
defined as ―all incidents involving an aircraft on the ground that results in damage or
wound, if there is no intention to fly, or, in the case of intention to fly, the causing factors
are related to ramp services and the movement of the aircraft or operation of its own
systems did not contribute to the event‖ (Comando da Aeronáutica, 2008a, p. 28).
Hence, both types of occurrence are specific types of incidents. Hereafter, grave
incidents and ground occurrences were treated as incidents in order to prevent
misinterpretations from those who are not familiar with the Brazilian denomination.
The relevance of incidents to aviation safety has never been perfectly understood.
Whereas accidents resulted in loss of life or aircraft and legal liabilities; incidents
resulted in injuries, damages, and limited liability. Additionally, incidents have been
more common and their investigation might produce better accident prevention results
than an accident investigation. The idea has been that the limited liability enabled a less
adversarial atmosphere during the investigation, unveiling relevant factors, mainly in the
human aspect (ICAO, 1993).
In the case of an occurrence of an aircrew error that resulted in an accident, the
investigation that took place involved a single individual, who may or may not be trained
in human factors. In addition, unlike the tangible and quantifiable evidence surrounding
mechanical failures, the evidence and cause of human errors were generally qualitative
and elusive. As a result, human factors investigations have traditionally focused on what
caused the accident, rather than why it occurred (Shappell & Wiegmann, 2003). Indeed,
accidents reports did not contain many human causal factors on which safety
recommendations could be based. The reports were only merely summaries of the
accidents or errors (ICAO, 1993). The data collected in human factors investigations was
34
also organized in databases. However, the databases lacked consistency and were useless
tools when applied to human factors investigations, prevention, and industry for
feedback. Furthermore, these problems within the human factors database have
prohibited the objective evaluation of most interventions. As a result, the overall rate of
accidents related to human error has remained high and constant over the last several
years (Wiegmann & Shappell, 2001a).
Human factors investigation. In the past, human factors investigations have
completely focused on determining the medical causes of death to assist in understanding
the mechanism of the accident. Medics, pathologists, and human engineering specialists
were the human factors experts. Their work was oriented by the Chapter 9, Part IV of the
ICAO's Manual of Aircraft Accident Investigation (ICAO, 1976). The continued
importance of human error in the genesis of aircraft accidents has created the necessity to
establish another approach to human factors investigations (Shappell & Wiegmann,
2001).
As a result, in 1993, the ICAO published the Circular 240-AN/144, titled
Investigation of Human Factors in Accidents and Incidents, to complement its Accident
Investigation Manual and to improve the ability to identify the involvement of human
factors in accidents and incidents. Based on Reason's (1990) model of human error and
on the Software, Hardware, Environment, and Liveware (SHEL model) (ICAO, 1993),
which addresses the importance of human interaction with the systems, the Circular 240AN/144 fostered the human factors investigations as a whole. Thereafter, the human
factors investigators were expected to be familiar with the psychological aspects of
human performance. A simple classification of factors was introduced, but these factors
were not supposed to be included in the accident Final Report issued by the national
35
aviation authority. Some checklists were included to help investigators rate the
importance of each factor (ICAO, 1993). Still, a comprehensive and useful framework
for identifying and analyzing human factors continued to elude investigators (Shappell &
Wiegmann, 2001).
Human factors investigation in Brazil. Authorized by law, CENIPA regulates
aircraft incident and accident investigations in Brazil. CENIPA’s investigation manuals
have to be followed by all entities that possess any kind of certificate issued by both the
Brazilian Air Force and the Brazilian National Civil Aviation Agency (ANAC)
(Comando da Aeronáutica, 2008b). Within the Brazilian Ministry of Defense, the
Brazilian Navy and the Brazilian Army have the opportunity to establish their own
internal procedures as long as they do not transgress the general guidance provided by
CENIPA.
As a continuing development in the investigative process, the human factors
investigations were divided by CENIPA into three aspects: (a) medical,
(b) psychological, and (c) operational. The objective of the investigation was to obtain
the factors involved in accidents and the levels of influence in the accident for each factor
discovered. All possible human factors were to be listed and defined to orient the
investigator within the process. The human factors reflected the situation or
psychological state of the crew during or before the accidents, but the cause of an error, if
one occurred, was not addressed. The human factors investigative group included an
Aviation Psychologist trained by CENIPA, or another certificated institution (Comando
da Aeronáutica, 2008c). In the accident Final Report, the human factors that contributed
to the accident were placed together with material factors as the Contributory Factors
36
lists. Each Contributory Factor was summarized based on the findings during the
investigation (Comando da Aeronáutica, 2008b).
Accidents Investigation Reports
In 1970, ICAO established the standard procedures for aircraft accident
investigations, including final reports and human factors investigation. The Contracting
States of the Chicago Convention were allowed to establish national regulation and
practices different from ICAO's standard, but these differences had to be published
according to the Chicago Convention Annex 15 (ICAO, 2006). Since then, the
development of new technologies and methods has been adopted by nations based on the
fact that these new improvements oppose neither the Chicago Convention Annex 13 nor
the ICAO's Manual of Aircraft Accident Investigation. The reports have been divided
into four sections: (a) factual information, (b) analysis, (c) conclusions, and (d) safety
recommendations (ICAO, 1976).
Varying according to the country, the differences in an accident Final Report were
more relevant in the conclusion section. The conclusion was established: (a) to indicate
which aspects of the flight were contributory to the accident and (b) to express the causes
of the accidents as a concise statement of the reason why the accident occurred (ICAO,
1976). The National Transportation Safety Board (NTSB) adopted the term Probable
Cause to explain the causes of the accident; the United Kingdom`s Air Accidents
Investigation Branch adopted the term Causal Factor, without further classification of the
factor; and the CENIPA, the Brazilian Regulatory Authority, adopted the term
Contributory Factor, classifying these factors into human or material factors (Comando
da Aeronáutica, 2008b). Within the Brazilian Navy, the SIPAAerM also included the
operational factor, which refers to the human performance in the activities related to the
37
flight. This factor includes the events associated with the preparation for the flight,
training, normal and emergency procedures, maintenance services, and air traffic control
(Marinha do Brasil, 2005a). The CENIPA established the operational factor as one of the
human factors sub-parts (Comando da Aeronáutica, 2008b).
Brazilian Navy accidents investigation and reports. Following the regulations
stated by CENIPA and ICAO, the SIPAAerM developed its own regulation to secure
safety operations in the Brazilian Navy. The peculiarities of Navy operations led to small
changes in the final report rules in which the investigator must enumerate all preceding
occurrences where the same contributory factors were present. This procedure was
included to evaluate the effectiveness of preceding safety recommendations that were
issued. Additionally, the operational factor that contributed to an accident or incident
was stated separately from human factors despite the fact that CENIPA human factors
classification encompasses the operational aspect (Marinha do Brasil, 2005a).
Furthermore, SIPAAerM stated that aircraft incidents and ground occurrences did
not need to have a complete final report. SIPAAerM established that these small
accidents would be reported by an Incident Summary and Ground Occurrence Summary.
These summaries would have to include the most relevant of the factual information
section and the analysis section in order to be simple and direct. The other sections were
not to be changed. However, as soon as an incident or ground occurrence investigation
brought to light important and potential prevention issues, the final report could be
applied for the benefit of aviation safety (Marinha do Brasil, 2005a).
At the same time, the investigative processes were established in different
manners due to the types of accidents. In the case of an accident, the investigative
process became more complex and involved numerous elements; a complete investigative
38
commission, with specialists in different areas became mandatory. The incidents and
ground occurrences were treated as a more simple investigative process and the Aviation
Safety Officer of the squadron to which the aircraft belonged was put in charge of the
process. Additionally, the Aviation Safety Officer was to be assisted by a small group of
specialists who integrated a simplified investigation commission. In both cases, an
aviation psychologist was included in the investigative process as a rule (Marinha do
Brasil, 2005a).
Previous Research Applying HFACS
In 2001, Shappell and Wiegmann used the HFACS framework outside the
military environment for the purpose of assessing the utility of this framework in civil
aviation. The database records of air carrier accidents between 1990 and 1998 were
studied and reviewed using the HFACS framework. They used an aviation psychologist
and a commercially-rated pilot to code the NTSB causal factors. No new causal factors
were created during the error-coding process. The reliability of the HFACS system was
assessed by calculating Cohen's Kappa; the result reflected a good level of agreement
between the two coders. The results proved the applicability of this new tool within
commercial aviation. Some of the error-factors within the HFACS framework were
never observed in the commercial aviation accident database (Shappell & Wiegmann).
In 2003, Shappell and Wiegmann used the HFACS to analyze the human errors of
general aviation Control Flight into Terrain (CFIT) accidents occurring between 1990
and 1998. They used five general aviation pilots recruited from Oklahoma City, as
subject matter experts. The pilots received 16 hours of HFACS training. More than
16,500 general aviation accidents were analyzed; 1,407 were classified as CFIT and then
compared with non-CFIT accidents using HFACS. The analysis revealed a number of
39
differences in the pattern of human error associated with CFIT accidents. Findings from
this study support many of the interventions identified by the CFIT Joint Safety Analysis
Team (JSAT) and Joint Safety Implementation Team (JSIT), permitting safety
professionals to better develop, refine, and track effectiveness of selected interventions
strategies (Shappell & Wiegmann).
In 2005, Scarborough, Bailey, and Pounds used the HFACS framework to analyze
Air Traffic Control (ATC) operational errors using the operational error causes to
separate groups of errors. They compared HFACS with other methods available, and
decided to apply HFACS to 10,754 operational error reports occurring between 1998 and
2002. Two ATC experts, with more than 15 years of experience, were trained in the
HFACS framework, and their classification was done, isolated from the other coder. The
coefficient Kappa calculated for their classifications indicated a high level of agreement.
As a result, HFACS proved to be a useful taxonomy for classifying the causal factors
associated with operational errors in ATC environment (Scarborough, Bailey, & Pounds).
That same year, Wiegmann, et al. (2005) used the HFACS framework to identify
the exact nature of human errors in general aviation and to assist in the generation of
intervention programs. They used 10 unanswered questions originated by the FAA and
other safety institutions concerning the underlying nature of human error in general
aviation as a guide to their study. The data were collected in the general aviation
accidents NTSB database from 1990 to 2000. Seven general aviation pilots were
recruited from the Oklahoma City area as subject matter experts. They received 16 hours
of training on the HFACS framework, including lecture and practice. Each accident was
assigned to two random pilots, and the discrepancies in the classification were reconciled.
Quality assurance was done using three human factors or aviation psychologist experts to
40
review the classification independently. Less than 4% of all causal factors were
modified. Analyzing the results, the researchers were capable of answering all the
questions asked of them and pointing out several ways to reduce the rate of fatalities in
general aviation accidents (Wiegmann et al.).
Still in 2005, Gaur applied HFACS to civil aircraft accidents at the Directorate
General Civil Aviation in India from 1990 to 1999. From the 83 accidents that occurred
in that period, only 48 reports were available for analysis. The author analyzed and
classified all 48 reports reading only the findings and recommendations part of them.
Another HFACS expert analyzed 15 of the reports using the same information. The
comparison of sample showed an agreement of 87% between both classifications. The
differences were discussed and a consensus was achieved. The researcher concluded that
the HFACS framework could be retrospectively applied to accidents reports in India
(Gaur).
In 2006, Detwiler et al. studied over 17,000 general aviation accidents to uncover
the types of human errors, identified by HFACS, which contributed to general aviation
accidents in Alaska and compared those results with the rest of the United States.
General aviation accident data from calendar years 1990 to 2002 were obtained from
databases maintained by the NTSB and the FAA. Six general aviation pilots, with more
than 1,000 flight hours, were recruited from Oklahoma City area, as subject matter
experts. They received 16 hours of training on the HFACS framework, including lecture
and practice. Each accident was assigned to two random pilots, and the discrepancies in
the classification were reconciled. There was no reliability or quality assurance
procedure. The results showed that there was no major difference between types of
human error in Alaska and the rest of United States, but a different approach must be
41
used to achieve the FAA desired level of safety for general aviation in Alaska (Detwiler
et al.).
That same year, Dambier and Hinkelbein (2006) applied the HFACS framework
in the analysis of 239 General Aviation (GA) accidents registered in Germany in 2004.
The accidents were required to have a German pilot to be included in the study,
regardless of the location of the occurrence. The methodology employed in the analysis
was not completely reported, but the results showed many similarities with previous
research using HFACS, mainly in the U.S. The researchers concluded that HFACS
seemed to be a valuable tool to analyze aviation accidents (Dambier & Hinkelbein).
Very similar research was conducted within the U.S. military services in 2006 by
Tvaryanas, Thompson, and Constable. They applied the HFACS framework in the
analysis of 221 remotely piloted aircraft mishaps from 1994 to 2003. Despite the absence
of human fatalities related to these accidents, the financial investments made in the
development and construction of these aircraft was sufficient reason for the analysis.
Two separate raters coded each mishap using the HFACS standards established by the
Department of Defense. The disagreements between their classifications were then
solved by the raters through a discussion of each case, but the agreement level was not
reported. The research revealed recurring factors at the organizational and supervisory
levels that needed to be addressed in order to improve the remotely piloted aircraft
viability (Tvaryanas, Thompson, & Constable).
Another study conducted in 2006 by Li and Harris, analyzed 523 accidents in the
Republic of China Air Force between 1978 and 2002 using the HFACS framework. The
goal was to determine relationships between pilot errors and higher organizational levels
within the HFACS framework. The coding process was done by an instructor pilot and
42
an aviation psychologist. Both were trained on the HFACS framework for 10 hours
before staring the process. Each accident received a code of 1 or 0 for each HFACS
factor to record its existence or non-existence. Cohen’s Kappa values varied from 0.44 to
0.83, with 14 HFACS factors achieving more than 0.60, which is considered substantial
agreement. Using the Goodman and Kruskall’s lambda to calculate the proportional
reduction in error, the researchers calculated the odd ratios of one HFACS factor being
associated with the presence of another HFACS factor. They detected clearly defined
paths that related error at the first level with inadequacies at the higher levels (Li &
Harris).
In 2007, Shappell et al. conducted another study to extend previous examinations
of aviation accidents to include specific aircrew, environmental, supervisory, and
organizational factors associated with two types of commercial aviation (air carriers and
commuter) accidents using HFACS. Commercial aviation accident data from the
calendar years 1990 through 2002 was obtained from databases maintained by the NTSB
and the FAA. Six pilots, with more than 1,000 flight hours, were recruited from
Oklahoma City area, as subject matter experts. All of the pilots were certified flight
instructors at the time they were recruited. They received 16 hours of training on the
HFACS framework, including lecture and practice. Each accident was assigned to two
random pilots, and the discrepancies in the classification were reconciled. Two human
factors experts were used to support the quality assurance and only 5% of all pilot-rater
classifications were modified. They concluded that their findings represented the
marriage of traditional demographic and human error analyses of commercial aviation.
The results also provided additional information for the development, implementation,
43
and quantifiable assessment of putative intervention and mitigation strategies (Shappell et
al.).
In 2008, Lenné, Ashby, and Fitzharris analyzed 169 GA crashes in Australia
between February 2002 and July 2004. The aim of this research was to examine
association of failures across the HFACS levels and to determine the likelihood of one
factor given the presence of a factor in a higher HFACS level. All three researchers were
experienced aviators and attended a training course specially developed to introduce
HFACS to them in order to conduct this study. Using a database to record demographic
information and HFACS classifications, each researcher analyzed all 169 accidents. The
inter-rater agreement observed was over 80%. Due to limited data on the accident
reports, the study limited the association analysis between HFACS levels 1 and 2. They
found that substandard personal readiness, physical or mental limitations, and adverse
mental state was associated with skill-based and decision errors. The researchers also
reported that lack of external influence factors in the HFACS might have been hindering
further analysis of accidents (Lenné, Ashby, & Fitzharris).
Following this research, in 2008, the Australian Transport Safety Bureau (ATSB)
funded research seeking to evaluate the use of HFACS as a predictive model. The
accident data encompassed 2,025 Australian aviation accidents reported to ATSB from
January 1993 to December 2003. In order to account for the external influences that
limited the previous analysis, this research included a fifth HFACS level, the outside
influence. This level was composed of: (a) maintenance issues, (b) airport/airport
personnel, (c) regulatory influence, (d) ATC action and issues, and (e) other personnel
involvement. Using logistic regression within and through the HFACS levels, the
research found 38 relationships, many of them consistent with previous research. Due
44
the amount of variation within the statistical model used, the study concluded that
HFACS might have limited effectiveness as a predictive framework. However, the
associations found in the study might help investigators to look for the associated factors
when contributing factors are found (ATSB).
Still in 2008, Li, Harris, and Yu conducted an analysis of 41 civil aviation
accidents from the Republic of China using HFACS. These accident reports were
obtained from the Republic of China Aviation Safety Council, covering from 1999 to
2006. The researchers were looking for the patterns in the roots of errors within civil
aviation. Two human factors specialists conducted the coding. Both were trained in
HFACS by an aviation psychologist in a three half-day course. The presence or absence
of each HFACS factor was evaluated and recorded for each accident using a binary
annotation. Inter-rater reliability was calculated using Cohen’s Kappa. In half the
factors, the Kappa values were over 0.40, which was regarded as being acceptable. The
associations between factors were calculated by the use of chi-square and Goodman and
Kruskal’s lambda methods. The researchers concluded that the mechanisms associated
with operational errors seem to be common between civil and military aviation in China.
Supervisory and organizational influence played a very relevant role in the paths to
failure detected by the study (Li, Harris & Yu).
In 2009, Majumdar, Mak, Lettington, and Nalder conducted a study regarding
helicopter accidents in New Zealand and United Kingdom encompassing 796 accidents
from 1986 to 2006. HFACS was used in the analysis of the operational failures detected
during the investigations limiting the HFACS application to 480 accidents. The
methodology in the application of HFACS was not reported. The study found remarkable
differences in the type of errors from the study conducted in 2005 by Wiegmann et al.
45
using GA accident in the U.S. The authors reported that the accident reports limited their
analysis of the organizational influence due to lack of investigative information
(Majumdar, Mak, Lettington & Nalder).
Summary
Since the technology evolution of aviation has reduced the likelihood of hardware
failures in aviation systems, safety and investigative organizations have tried to develop a
tool to address the highly complex, and often nebulous, issue of human error (Shappell &
Wiegmann, 2003). Despite the lack of regulatory standards concerning the human factors
investigative process and the variety of approaches applied in different countries,
including the accident final report’s rules and human factors’ classification, an increasing
concern exists about the importance of human errors in aviation accidents and incidents.
Due to the force of law, Brazilian Naval Aviation followed Brazilian Air Force regulation
and human factors classification system, which did not follow a specific framework in the
analysis of the human errors involved in accidents and incidents. Using the Reason's
(1990) model of accidents causation as a background, the HFACS was developed in the
military environment to expose human causes of accidents. Additionally, the model has
proven its applicability in many different aviation environments across the world.
Research Questions
1. What are the most common chains of HFACS factors that were identified as
contributory causes in the Brazilian Naval Aviation incidents?
2. What are the most common HFACS factors that were identified as
contributory causes in the Brazilian Naval Aviation incidents?
46
Chapter III
Methodology
Until recently, the Brazilian Navy has not used any method to analyze the
underlying causes of human error that contributed to aviation incidents. Limited
instruments have been implemented to address this significant aspect of aviation
incidents in the Brazilian Naval Aviation due to some limitations imposed by the
regulations established by the Brazilian Air Force. In this study, the HFACS framework
was employed in a quantitative method to analyze the human factors involved in these
incidents.
Research Approach
This descriptive study used historical incident data. Isaac and Michael (1995)
have defined descriptive research as the type that is primarily concerned with describing
systematically the facts and characteristics of a given population, factually and
accurately, the emphasis being on describing, rather than on judging or interpreting.
Design and procedures. After the SIPAAerM granted permission to conduct
preparatory research, the researcher read all of the aviation incidents summaries of the
Brazilian Navy since 1997 until 2010. The initial step in the research procedure was to
establish the number of summaries that contained appropriate investigative information
regarding human factors that would permit the application of the HFACS as a
classification taxonomy. The researcher determined that all incident summaries issued
before 1999 were not suitable for classification analysis due to lack of information and
different human factors approach that was in use at that time by the SIPAAerM. During
this procedure, the researcher was also able to identify the incident summaries post-1999
in which human error was not cited as a contributory factor and they were eliminated
47
from this study. The researcher numbered all remaining summaries that were included in
this study as a simple means of organizing them. The second step of this research
procedure was to identify which summaries could be classified using the HFACS-MEDA
framework according to the relevant role of the maintenance workers in the incident.
The researcher designed an HFACS coding worksheet in Microsoft Excel® to use
in coding the incident summaries. This worksheet was developed to collect demographic
information about the incidents and to record the HFACS classification in two different
manners:
1. chain of factors
2. entire framework analysis.
The chain of factors records was established to identify the different chain of
HFACS factors that contributed to each incident, whereas the entire framework analysis
was established to identify the existence or non-existence of each HFACS factor in each
incident. The chain of factors record was limited to one factor per HFACS level. If more
than one factor was identified in the same level, two chains of factors were recorded to
reflect the appropriate influence that each factor had on the factor in the lower levels of
the HFACS framework. The entire framework analysis was the summation of all chains
of factors in each incident using a binary annotation.
Due to time and cost constraints, the researcher coded all incidents using the
tabular HFACS coding worksheet. Each human causal factor of the incidents was coded
using the HFACS framework, identifying the chains of factors. After the chain of factors
coding was completed, the researcher completed the entire framework analysis. If the
incident summary did not have enough investigative information to support an
appropriate classification despite the initial evaluation conducted, the researcher marked
48
the first HFACS level in the chain of factors analysis with a ―FAIL‖. The entire
framework analysis was not completed for any ―FAIL‖ cases.
The theoretical difference between routine violations and exceptional violations,
in regards to their acceptance by the organization, led the researcher to code both HFACS
factors as a violation. The search for a reason to code this type of unsafe act might have
resulted in skewed data. In previous studies using historical accident data, researchers
faced with this concern did not try to distinguish between the two types of violations
(Shappell & Wiegmann, 2003; Wiegmann et al., 2005; Dambier & Hinkelbein, 2006; Li
& Harris, 2007; Shappell et al., 2007; Lenné, Ashby, & Fitzharris, 2008; Li, Harris, &
Yu, 2008;).
The coding process was done twice, with each process separated by 15 days. In
both processes, coding followed a random order of incidents in accordance with two
random sequences taken from a random order generator. After the first coding round, the
researcher saved the HFACS coding worksheet into a file and stored it in an appropriately
designated folder. Likewise, the incident summaries used in the first round were stored
in a separate binder to prevent the researcher from reading his own annotations written
during the first round when repeating the process for the second round of analysis. The
second coding round followed the same procedure in regards to the HFACS coding
worksheet and the incident summaries as that of the first round.
The next step in the procedure was to detect and solve any disagreements between
the two coding rounds. After identifying the disagreements, the researcher read the
related incident summaries once more and then defined what HFACS coding should be
accepted as the final coding. The collection of final classification data were then used in
the descriptive analysis.
49
Population
The population for this study was the Brazilian Naval Aviation incident
summaries from 1999 to 2010 in which the incident investigators identified at least one
human error as a contributory factor in the incidents, totaling 157 cases. All of these
summaries were issued by the SIPAAerM and were written in Portuguese.
Sources of the Data
The SIPAAerM incidents database was the source of all data for this study. The
researcher was granted permission (see Appendix B) to use this confidential data
specifically in this study. Table 1 defines the SIPAAerM data. Using the HFACS
framework (see Appendix C), the researcher classified the SIPAAerM data into 18
HFACS factors divided into four levels (see Table 2), thus, comprising the chain of
factors analysis. Appendix D presents a sample of the result of the chain of factors
analysis.
50
Table 1
Variables Collected from SIPAAerM Summaries
Variable
Ordinal
Number
Summary
Number
Incident
Type
Measurement Definition
Level
Nominal
A sequential number defined by the researcher.
Nominal
The number issued by SIPAAerM to each summary.
Nominal
Indicates if the event is a Ground Occurrence or an
Aeronautical Incident, according to SIPAAerM
classification.
Operator
Nominal
Indicates the Squadron that operates the aircraft involved in
the incident.
Model
Nominal
Indicates the aircraft model involved in the incident.
Year
Nominal
Indicates the year in which the incident occurred.
Location
Nominal
Indicates the place in which the incident occurred. Divided
into four groups: SBESa, Airports, Ships, and Other.
Flight Phase Nominal
Indicates the phase of the flight in which the incident
occurred, according to SIPAAerM definitions.
Costs
Scale
Indicates the total cost of the incident, including
investigation costs.
a
SBES is the Brazilian Navy Air Naval Base ICAO Airport Code.
Table 2
Variables Used in the Chain of Factors Analysis
Variable
Measurement Definition
Level
Typea
Nominal
Indicates if the event is related to Flight Crew or
Maintenance unsafe act.
a
Level 1 Factor
Nominal
Indicates the factor identified within HFACS Level 1.
Level 2 Factor
Nominal
Indicates the factor identified within HFACS Level 2.
Level 3 Factor
Nominal
Indicates the factor identified within HFACS Level 3.
Level 4 Factor
Nominal
Indicates the factor identified within HFACS Level 4.
Type variable was included to define whether each incident was analyzed using the
HFACS or the HFACS-MEDA codes guidance.
51
To accomplish the entire framework analysis, the researcher used the HFACS
framework to identify the presence or the absence of each HFACS factor. Each HFACS
factor received a notation of one to indicate the presence of the factor or zero to indicate
the absence of the factor. All HFACS factors were defined in Chapter II and a sample
result of the entire framework analysis has been provided in Appendix E.
Reliability
This research design followed the approach used by HFACS designers in
applying the framework in different environments (Wiegmann & Shappell, 2001a;
Shappell & Wiegmann, 2003; Wiegmann, Faaborg, Boquet, Detwiler, Holcomb, &
Shappell, 2005). These procedures were followed in this study to ensure reliability.
Expert reliability. Because the researcher was required to code reports written in
Portuguese, the primary language of the researcher, it was deemed necessary to have
HFACS experts check his coding process accuracy, because English is his second
language. Two HFACS experts achieved this check through the classification of two
sample incident summaries. The researcher translated two random incident summaries
into English and then presented the information to the experts. A discussion persuaded to
the appropriate codes that should be applied to each incident. The researcher presented
his final classification after the group achieved an agreement on the codes for each
summary. This way, the experts were not influenced by the researcher’s classification.
Then the experts provided the researcher with feedback and instructions as to what
should be reanalyzed in the HFACS coding worksheet.
This reliability check also provided the researcher with the opportunity to explain
the peculiarities of the Brazilian Naval Aviation organization and its climate that makes
its environment different from what the experts are familiar within the U.S. Although
52
these differences guided the researcher analysis of the data, both the experts validated the
researcher’s procedures.
During the expert check, the researcher also presented preliminary results from
the final entire framework analysis. Maintenance cases, flight crew cases, and reliability
results were presented in a single table. The experts suggested that the researcher
reanalyze all maintenance-related summaries in which a decision error was identified
because of the high number of this factor in the researcher’s analysis. The results of this
revisit were recorded by changes in the second round analysis. These results were also
considered the final classification because that was the third time each summary had been
analyzed and the researcher had been provided further guidance from the HFACS
experts.
Intra-rater reliability. Intra-rater reliability is the type of reliability assessment
that deals with the degree of stability of instrument scores for one rater across two or
more times (Portney & Watkins, 2000). Cohen’s Kappa is extensively applied to
measure inter-rater reliability and can be applied to intra-rater reliability as well. Table 3
presents the standard interpretation of the coefficient values in terms of agreement level
in accordance with Viera and Garret (2005). Using the Cohen’s Kappa coefficient, the
HFACS framework application proved to be reliable in a previous analysis of commercial
aviation accidents in which a 0.71 value was obtained (Wiegmann & Shappell, 2001a).
Li and Harris (2006), Li, Harris and Yu (2008), and Rashid, Place, and Braithwaite
(2010) also reported Cohen’s Kappa coefficients in their research.
53
Table 3
Cohen’s Kappa Coefficient Interpretation
Coefficient
κ<0.2
0.21< κ<0.4
0.41< κ<0.6
0.61< κ<0.8
κ >0.8
Agreement Level
Slight agreement
Fair agreement
Moderate agreement
Substantial agreement
Almost perfect agreement
Validity
SIPAAerM is the organization within the Brazilian Navy that is responsible for
issuing all aviation accident reports and incident summaries (Marinha do Brasil, 2005a).
Due to law requirements, SIPAAerM has to comply with the procedures established by
CENIPA, the Brazilian Air Force accident investigation organization that issues all
procedures and regulations for Brazilian aviation. SIPAAerM issued and approved all
summaries used in this study. Thus, the data was considered valid.
Treatment of the Data
The researcher arranged the HFACS coding worksheet to create a dataset that
could be analyzed through statistical methods. The dataset contained only quantitative
variables. The results from two coding processes were compared and the disagreements
were resolved by the researcher by reviewing the related summaries. The final coding
result was used in all of the statistical procedures in this study. The confidence level for
all tests of significance was 95%, regardless of the use of parametric or non-parametric
statistics. This decision followed the previous research that applied the HFACS
framework (Shappell & Wiegmann, 2001; Shappell & Wiegmann, 2003; Scarborough,
54
Bailey & Pounds, 2005; Wiegmann, Faaborg, Boquet, Detwiler, Holcomb & Shappel;
2005, Detwiler et al.; 2006; Shappell et al., 2007).
Descriptive statistics. The researcher used figures to describe all nominal
variables created to collect the HFACS coding in the chain of factors analysis and the
entire framework analysis. The ratio variable, cost, was described in a table depicting the
mean, standard deviation, maximum, minimum, and count.
Reliability testing. The reliability of the coding process was measured by the use
of intra-rater reliability. Cohen’s Kappa coefficient for each HFACS factor and for the
whole framework between the two separate coding processes was calculated. Gaur
(2005) reported agreement levels within each HFACS factor as a measurement of how
many times the reports had to be reanalyzed in order to achieve a final coding. Despite
its weakness as a statistical method, the agreement level in percentage for each HFACS
factor and for the whole framework was also calculated.
55
Chapter IV
Results
From the 157 incident summaries initially selected by the researcher, only 141
were completely analyzed. Some of the discarded summaries contained limited
investigatory information regarding the incident. Others encompassed more than one
incident, making it difficult for the researcher to identify the contributory factors related
to each specific incident included in the summary. The results of the 141 summaries
analyzed in this study are presented in the following paragraphs.
Descriptive Statistics
Incidents’ demographic variables. This study classified 141 Brazilian Naval
Aviation incidents. Figures 8, 9, 10, 11, 12, and 13 describe all nominal variables
collected from SIPAAerM incident summaries.
Aeronautical Incident
Ground Occurrence
n=33 (23%)
n=108 (77%)
Figure 8. Description of incident type variable.
56
40
34
35
30
24
25
20
21
20
15
11
11
12
10
5
3
5
0
Figure 9. Description of operator variable.
35
29
30
24
25
20
14
15
10
9
11
11
12
6
5
0
Figure 10. Description of model variable.
25
57
25
23
21
20
14
15
12
9
10
6
5
6
9
9
13
9
7
3
0
Figure 11. Description of year variable.
n=14
(10%)
n=17 (12%)
SBES
Airports
n=74 (52%)
n=36 (26%)
Figure 12. Description of location variable.
Ships
Other
58
30
27
25
25
22
20
14
15
10
7
5
1
1
2
2
3
3
3
3
3
4
8
9
4
0
Figure 13. Description of flight phase variable.
Cost was the only ratio variable collected in this study. As presented in Table 4,
which shows the descriptive statistics for the variable, an outlier was skewing the data.
The results with and without the outlier are presented in order to provide a better
understanding of the cost involved in the incidents. Figures 14 and 15 present the
associated histograms.
Table 4
Descriptive Statistics for Cost
Cost
Mean
Standard Deviation
Minimum
Maximum
N
Entire dataset
Value
25,766.60
80,105.46
20.00
800,000.00
141
Dataset without outlier
Value
20,236.36
46,040.70
20.00
257,279.90
140
59
Figure 14. Histogram of the incidents’ cost variable, entire dataset.
Figure 15. Histogram of the incidents’ cost variable, dataset without outlier.
Chain of factors analysis variables. This study identified 159 different chains of
factors that contributed to the 141 incidents analyzed. Figure 16 describes the type
variable, and Table 5 and Figures 17, 18, 19, and 20 describe the variables within each of
the four HFACS levels.
60
n= 63
(40%)
Flight Crew
Maintenance
n= 96
(60%)
Figure 16. Description of the type variable.
Table 5
Chain of Factors Variables
Level
Level 1
Level 2
Level 3
Level 4
Factor
Skill-Based Error
Decision Error
Violation
Perceptual Error
Adverse Mental State
NIL
Crew Resource Management
Physical /Mental Limitations
Technological Environment
Adverse Physiological State
Physical Environment
Personal Readiness
Inadequate Supervision
NIL
Planned Inappropriate Operations
Failed to Correct Problem
Supervisory Violations
NIL
Organizational Process
Resource Management
Organizational Climate
Note. NIL means that there was no factor coded on that level.
Count
72
62
18
7
65
37
24
22
5
4
1
1
59
49
25
13
13
107
26
17
10
Percentage
45.3
39
11.3
4.4
40.9
23.3
15.1
13.8
3.1
2.4
0.7
0.7
37.1
30.8
15.7
8.2
8.2
66.7
16.4
10.7
6.2
61
50.0
45.3
45.0
39.0
40.0
Percentage
35.0
30.0
25.0
20.0
15.0
11.3
10.0
4.4
5.0
0.0
Skill-Based Error Decision Error
Violation
Perceptual Error
Percentage
Figure 17. Description of Level 1 variable.
45.0
40.0
35.0
30.0
25.0
20.0
15.0
10.0
5.0
0.0
40.9
23.3
15.1
13.8
Figure 18. Description of Level 2 variable.
3.1
2.4
0.7
0.7
62
45.0
40.0
37.1
Percentage
35.0
30.8
30.0
25.0
20.0
15.7
15.0
10.0
8.2
8.2
Failed to
Correct
Problem
Supervisory
Violations
5.0
0.0
Inadequate
Supervision
NIL
Planned
Inappropriate
Operations
Figure 19. Description of Level 3 variable.
80.0
70.0
66.7
Percentage
60.0
50.0
40.0
30.0
16.4
20.0
10.7
6.2
10.0
0.0
NIL
Organizational
Process
Resource
Management
Organizational
Climate
Figure 20. Description of Level 4 variable.
Besides the analysis of each variable within the four HFACS levels, the
researcher sought the most common chains of factors. Table 5 presents the results of this
analysis with all chains that were identified more than once.
63
Table 6
Most Common Chain of Factors
Level 1 Factor
Level 2 Factor
Level 3 Factor
Level 4 Factor
Count
Skill-Based Error
NIL
Inadequate Supervision
NIL
10
Skill-Based Error
Adverse Mental State
NIL
NIL
9
Skill-Based Error
CRM
NIL
NIL
8
Decision Error
CRM
NIL
NIL
5
Skill-Based Error
Physical /Mental Lim.
NIL
NIL
5
Decision Error
Adverse Mental State
Inadequate Supervision
NIL
4
Decision Error
Adverse Mental State
Inadequate Supervision
Org. Process
4
Decision Error
Adverse Mental State
NIL
NIL
4
Decision Error
NIL
Inadequate Supervision
NIL
4
Decision Error
NIL
NIL
NIL
4
Violation
Adverse Mental State
Inadequate Supervision
NIL
4
Skill-Based Error
Adverse Mental State
Inadequate Supervision
NIL
4
Skill-Based Error
NIL
NIL
NIL
4
Decision Error
Adverse Mental State
Inadequate Supervision
Org. Climate
3
Decision Error
Adverse Mental State
Supervisory Violations
Org. Process
3
Skill-Based Error
Physical /Mental Lim.
Inadequate Supervision
NIL
3
Skill-Based Error
Physical /Mental Lim.
NIL
Resource Management
3
Skill-Based Error
Technological Env.
Inadequate Supervision
NIL
3
Decision Error
Adverse Mental State
Planned Inap. Operations
NIL
2
Decision Error
CRM
Planned Inap. Operations
NIL
2
Decision Error
CRM
Supervisory Violations
NIL
2
Decision Error
NIL
Supervisory Violations
NIL
2
Perceptual Error
Adverse Mental State
Inadequate Supervision
NIL
2
Perceptual Error
Adverse Mental State
Planned Inap. Operations
NIL
2
Skill-Based Error
CRM
Planned Inap. Operations
NIL
2
Note. Technological Env. stands for Technological Environment and Planned Inap.
Operations stands for Planned Inappropriate Operations.
Entire framework analysis variables. This study identified 145 different entire
framework cases that contributed to the 141 incidents analyzed. The difference from the
64
number of incidents analyzed resides within the incidents in which both maintenance and
flight crew played significant roles in the sequence of events that led to the incidents. In
those cases, each type of framework was analyzed and recorded separately. Table 7 and
Figure 21 describe the percentage of occurrence of all HFACS factors recorded in this
analysis. The factors are organized in ascending order of total occurrence, which is the
summation of maintenance and flight crew occurrences. Each type of occurrence, flight
crew and maintenance, is presented in different bars in order to allow a comparison
between them.
Table 7
Entire Framework Analysis Variables
Factor
Personal Readiness
Physical Environment
Technological Environment
Adverse Physiological State
Perceptual Error
Organizational Climate
Failed to Correct Problem
Supervisory Violations
Resource Management
Violation
Physical/Mental Limitations
Planned Inappropriate Operations
Organizational Process
Crew Resource Management
Inadequate Supervision
Decision Error
Adverse Mental State
Skill-Based Error
Flight Crew
Count Percentage
0
0
1
3
4
4
2
1
3
1
10
17
8
22
8
20
22
30
0.00
0.00
0.69
2.07
2.76
2.76
1.38
0.69
2.07
0.69
6.90
11.72
5.52
15.17
5.52
13.79
15.17
20.69
Maintenance
Count Percentage
1
1
4
2
3
5
10
12
13
16
11
7
16
3
50
39
40
35
0.69
0.69
2.76
1.38
2.07
3.45
6.90
8.28
8.97
11.04
7.59
4.83
11.03
2.07
34.48
26.90
27.59
24.14
Total
Count Percentage
1
1
5
5
7
9
12
13
16
17
21
24
24
25
58
59
62
65
0.69
0.69
3.45
3.45
4.83
6.21
8.28
8.97
11.03
11.73
14.48
16.55
16.55
17.24
40.00
40.69
42.76
44.83
65
Maintenance
Personal Readiness
0.7
0.0
Physical Environment
0.7
0.0
Flight Crew
Technological Environment
2.8
0.7
Adverse Physiological State
1.4
2.1
Perceptual Error
2.1
2.8
Organizational Climate
3.4
2.8
6.9
Failed to Correct Problem
1.4
8.3
Supervisory Violations
0.7
9.0
Resource Management
2.1
11.0
Violation
0.7
7.6
6.9
Physical/Mental Limitations
11.0
Organizational Process
5.5
4.8
Planned Inappropriate Operations
11.7
2.1
Crew Resource Management
15.2
34.5
Inadequate Supervision
5.5
26.9
Decision Error
13.8
27.6
Adverse Mental State
15.2
24.1
Skill-Based Error
20.7
0.0
5.0
10.0
15.0
20.0
25.0
Percentage of Occurrence
Figure 21. Description of entire framework analysis variables.
30.0
35.0
40.0
66
Reliability Testing
The reliability test was conducted by calculating the agreement and the Cohen’s
Kappa coefficient between the first and second analysis round. Table 8 presents the
reliability results for each HFACS factor and for the whole framework.
Table 8
Reliability Results
HFACS HFACS Factor
Level
1
2
3
4
All
Skill-Based
Perceptual
Decision-Making
Violation
Physical Environment
Technological Environment
Adverse Mental State
Adverse Physiological State
Physical/Mental Limitations
Crew Resource Management
Personal Readiness
Inadequate Supervision
Planned Inappropriate Operations
Failed to Correct Problem
Supervisory Violations
Resource Management
Organizational Climate
Organizational Process
All HFACS Factors
Number of
Occurrences
Agreement
Percentage
Cohen’s
Kappa
65
7
59
17
1
5
62
5
21
25
1
58
24
12
13
16
9
24
424
83.4%
96.6%
79.3%
97.2%
100.0%
99.3%
93.1%
99.3%
91.7%
93.8%
99.3%
85.5%
93.1%
91.7%
95.2%
92.4%
97.9%
92.4%
93.6%
0.66
0.72
0.59
0.68
1.00
0.89
0.86
0.91
0.68
0.75
0.99
0.70
0.74
0.46
0.67
0.58
0.83
0.67
0.75
67
Chapter V
Discussion, Conclusions, and Recommendations
This descriptive study used historical data to detect the underlying causes of
human error that contributed to aviation incidents within Brazilian Naval Aviation. The
HFACS framework, which has proven effective in this task in many other countries, was
utilized to achieve this study goal.
Discussion
Identifying strengths and weaknesses within an organization might help develop
recommendations for improving processes, culture, and resource management to target
safety concerns. In the specific case of incidents related to human error, the lack of a
comprehensive framework that allows for understanding the contributing factors to each
incident might hinder the implementation of an appropriate mitigation strategy.
Moreover, the aggregate data from a collection of incidents might provide the necessary
insight to develop a better safety mitigation approach.
Each incident analyzed in this study seemed to be, at least on the surface,
reasonably unique. The application of the HFACS framework provided the trends in
specific types of errors and in the associated latent failures within the organization. This
data might be used to improve Brazilian Naval Aviation accident prevention policy and
develop better incidents investigatory processes.
Descriptive statistics. The researcher analyzed and coded 141 incident
summaries, rendering 159 chains of factors and 145 entire framework cases. This
difference occurred because the researcher included as many chains of factors as
necessary to reveal all of the factors associated with each incident. In 14 incidents, there
68
were two chains of factors identified by the researcher. Additionally, in four cases, both
maintenance and flight crew committed unsafe acts, and the appropriate codes were
recorded for those incidents, but used different chains of factors and entire framework
cases to avoid skewing the data in both types of analysis.
Incidents’ demographic. The demographic information of the 141 incident
summaries was analyzed and the results presented some very relevant information. The
following paragraphs discuss each variable in regards to the relevant information its
analysis reveals.
Type of incident. The most relevant difference between a ground occurrence and
an aeronautical incident relies on the presence or absence of the intention to fly. Ground
occurrences accounted for 23% of the overall incidents analyzed in this study. In two
cases, the flight crews’ unsafe acts were reported as contributory causes in ground
occurrences. This is an interesting discovery because many pilots believe their
responsibility ends when the aircraft lands, the engine is shut down, and the aircraft has
been left to the maintenance crew to prepare for the next flight. In both cases, flight crew
committed decision errors associated with CRM failures. This means that decisions
taken by the flight crew in flight, in a poor CRM environment, influenced aircraft safety
after landing.
Operator and model. Before starting the discussion in regards to these variables,
it is important to present actual information in regards to the Brazilian Naval Aviation
fleet composition. Table 9 presents the unofficial data collected from two reliable
websites.
69
Table 9
Brazilian Naval Aviation Fleet Composition
Squadron
Aircraft
Model
Aircraft
Type
EsqdHU-1
UH-12
UH-13
UH-14
UH-12
IH-6B
UH-12
IH-6B
AH-11A
SH-3A
SH-3B
AF-1
AF-1A
Helicopter
Helicopter
Helicopter
Helicopter
Helicopter
Helicopter
Helicopter
Helicopter
Helicopter
Helicopter
Airplane
Airplane
EsqdHU-2
EsqdHU-3
EsqdHU-4
EsqdHU-5
EsqdHI-1
EsqdHA-1
EsqdHS-1
EsqdVF-1
Number
First
Flight
Airborne Systems
Complexity a
9
8
7
6
4
3
16
13
7
4
20
3
1974
1979
1977
1974
1962
1974
1962
1971
1959
1959
1954
1954
Low
Low
Medium
Low
Low
Low
Low
High
High
High
Medium
Medium
Note. Data collected from Helicopter History Site at www.helis.com and from Base
Militar Web Magazine at www.alide.com.br. a The complexity is the researcher’s
subjective classification, based on 9 years of experience on the Brazilian Navy as a Naval
Aviator.
Four out of nine squadrons represented more than 70% of the overall incidents
analyzed in this study. Not surprisingly, the squadron that had the highest number of
human error related incidents is the squadron that operates the oldest aircraft in the
Brazilian Naval Aviation inventory, the EsqdHS-1. These aircraft have been deemed one
of the most complex in terms of airborne systems in active duty. The squadron that had
the second highest number of incidents, the EsqdHA-1, operates the most modern
helicopter model in Brazilian Naval Aviation, AH-11A, and it also employs complex
airborne systems. The operation of complex systems seems to have played a relevant
role in aircraft maintenance because maintenance-related incidents accounted for 73%
and 70% in the EsqdHS-1 and EsqdHA-1, respectively. Both percentages were higher
than the 60% of maintenance-related incidents found in the chain of factors analysis. The
70
squadron with the third highest number of incidents, the EsqdHU-1, was the squadron
with the largest aircraft inventory, but it operates simpler aircraft than the first two
squadrons. In this squadron, which accounts for more flight hours than the first two and
operates the UH-12 and UH-13 models, maintenance-related human errors were 56%.
This result was reasonably similar to the chain of factors analysis distribution previously
cited. The fourth squadron in this top-four group was the EsqdVF-1, which is the only
squadron that operates airplanes. This squadron is also the youngest Brazilian Naval
Aviation squadron, with only 13 years in operation, operating the AF-1 and AF-1A
models. The adaptation from the helicopter environment to the airplane environment
seems to have played a relevant role in maintenance operations because it accounts for
75% of the incidents in this squadron. This squadron operates aircraft designed in the
1950’s. As with conclusions from the EsqdHS-1 case, aircraft age might have played an
important role in the high percentage of maintenance-related human errors.
Year. Almost one third of the overall incidents analyzed in this study occurred
between 2002 and 2003. Additionally, 61% of all incidents occurred before 2004. The
researcher found this result might be related to the SIPAAerM prevention policy and
mitigation strategies implemented in the last decade, which improved the relevance and
understanding of human factors in aviation. In 2002, the PPAA introduced the concept of
human error (Marinha do Brasil, 2002), and in 2003, the PPAA discussed the relevance
of an appropriate human factors approach to decrease the rates of incidents and accidents.
At that time, the aviation psychologist and aviation physician roles were emphasized as
relevant practices to reduce the occurrence of safety events. However, the reduced
number of these professionals hindered the effectiveness of this approach (Marinha do
71
Brasil, 2003). Notwithstanding these problems, the number of human errors was reduced
since 2003.
In 2005, the PPAA defined the differences between human error and violation,
and established the psychological oversight of all flight crews and maintainers as the best
mitigation strategy to reduce the influence of human factors in accidents and incidents.
They determined aviation psychologists should work in each squadron, flying with flight
crews and observing maintenance tasks, in order to understand the environment and
develop human factors strategies to cope with the deficiencies faced by squadron
personnel (Marinha do Brasil, 2005b). This approach resulted in a further reduction in
the number of human errors, except for 2008, when this number returned to 2004-2005
levels.
Location. Since six out of nine squadrons operate from the Brazilian Navy Air
Naval Base (SBES), it is no surprise that this location accounts for 52% of the incidents
analyzed in this study. The relevant aspect regarding the location of incidents location is
that ships and other locations, which encompass unprepared landing facilities, account for
only 22% of the human errors. This might be a result of improved situation awareness by
flight crew and maintenance personnel when operating at these locations, because they
are a more challenging environment than naval air base facilities. This difference in
situation awareness is probably a consequence of the utilization of ORM strategies that
were introduced in 2002 by the PPAA, and were reinforced by the second edition of the
Aviation Safety Manual (Marinha do Brasil, 2002; Marinha do Brasil, 2005a). However,
airports other than SBES account for 26% of incidents. This result contradicts the above
understanding in regards to improved situation awareness outside naval air base facilities.
72
A possible explanation might be that the challenging environment faced in ships also
creates complacency when operations occur in less challenging environment, like
airports. In this case, flight and maintenance crews understood the challenges with ship
operations, but were perhaps complacent while operating at airports due to their
perceived safer or less challenging environment.
Flight phase. The accepted claim that take-off and landing operations are the
most dangerous phases of flight was not confirmed in this study, but not without a caveat.
The second largest flight phase was not stated, which was utilized when the summary did
not contain the flight phase information. This fact happened in all summaries issued by
SIPAAerM in 1999 and 2000, encompassing 25 summaries, because the first edition of
the Aviation Safety Manual did not required the flight phase to be included in incident
summaries (Marinha do Brasil, 2000). Nonetheless, those summaries were not excluded
from the analysis because they contained enough investigatory information to allow for
the coding process. Consequently, flight phase data might be skewed since the number of
times not stated occurred is enough to affect the data analysis significantly.
The most common phase of flight in which incidents happened was ground, from
which two were aeronautical incidents, one involved flight crew human error, and the
other 24 were ground occurrences in which maintenance personnel committed human
errors. This result by itself represents the weight the actions of maintenance personnel
had in the unsafe acts detected by this study. Moreover, the third most common phase of
flight was cruise, from which 13 were related to maintenance unsafe acts and nine to
flight crew unsafe acts. Despite the idea that incidents in cruise flight are highly related
73
to flight crew errors, this also result reinforces the weight of maintenance personnel
unsafe acts in this study.
Even though landing as a flight phase defined by SIPAAerM was only the fourth
most common flight phase in this study, the actual number of cases under this category
would increase depending on the criteria utilized. By adding rolling after landing,
landing pattern, and final approach to the landing category, this new category would
equal cruise as the third most common flight phase. Thus, despite the need to cluster four
phases, as defined by SIPAAerM, to achieve a final landing category, the relevance of the
landing phase in incidents occurrence was not completely refuted. The same did not
happen with take-off, the sixth most common flight phase in this analysis, and no
reasonable clustering could make it relevant.
Another relevant result was the presence of two engine-related categories within
the seven most common ones. By adding engine start and engine check categories, the
number of occurrences was greater than the landing category by SIPAAerM definitions.
Twelve incidents of this new category were maintenance-related incidents, three were
flight crew incidents, and one involved both maintenance and flight crew. Again,
maintenance-related human error played a relevant role in a category that could be seen
as flight crew related.
Cost. The financial loss associated with an aviation accident is one of the most
relevant factors to encourage organizations to invest in accident prevention programs. In
the case of military branches, regardless of the country, this financial burden might not be
easily perceived because these organizations do not strive for profit. This phenomenon
seems to hide the real financial strain aviation accidents and incidents impose on military
74
branches. The cost of the 141 incidents analyzed in this study totaled $3,633,090.00,
ranging from $20.00 to $800,000.00.
This skewed distribution was caused by two main factors. First, the incident
summaries issued by SIPAAerM before 2001 did not include the investigation cost,
which represent an additional financial burden to the organization that cannot be
disregarded. Second, an engine-related incident in 2007 forced an engine overhaul,
which cost $800,000.00 due to the need to send the item to the manufacturer’s facility
overseas. This outlier skewed the mean and the standard deviation in the overall dataset
in such a way that another calculation was done without that outlier case. Even though
the new results seem to be a more accurate representation of the cost distribution, the
variance was still very large, which means that the severity of the incidents analyzed in
this study was very wide. A possible use of this information would be to focus mitigation
strategies in the more expensive incidents, but a deeper analysis of the relationship of
human error and cost would have to be accomplished before making any further
conclusions or decisions.
Chain of factors analysis. This analysis was designed to fulfill a HFACS
weakness reported in a review of the aviation human factors taxonomies conducted by
Beaubien and Baker (2002). They stated that the HFACS framework does not identify
the chain of events, making it complex to separate causes from effects. Even though the
researcher believes that the HFACS was not created with the intention to establish the
sequence of causes and effects, but to detect the latent and active failures that led to an
accident, a decision was made to include a specific coding procedure in this study to
achieve the chain of events analysis.
75
The 141 incident summaries rendered 159 chains of factors; eighteen incidents
contained two chains of factors. These parallel chains of factors represented the nonlinearity characteristic of the HFACS framework, which could be seen as a weakness as
well as strength, depending on how the framework was utilized to analyze the data. One
HFACS factor in a higher HFACS level might cause one or more factors in the lower
levels, as well as none. This non-linearity was in complete agreement with Reason’s
(1990) Swiss Cheese theory that postulates more than one latent failure might be present
in an accident. The way the HFACS framework was set up might mislead an untrained
person. Thus, use of the chain of factors analysis without a previous understanding of
this HFACS framework characteristic might have compromised the results of the
analysis, skewing the data or keeping relevant HFACS factors out of the final codes
applied to an incident.
Again, the data revealed that maintenance-related human error were more
common than flight crew-related human error within Brazilian Naval Aviation.
Maintenance errors represented 60% of the chain of factors, whereas flight crew errors
represented 40%. Despite the fact that the researcher found no other research that
compared human error in maintenance and flight crew environment, the results of this
study should be carefully analyzed before generating the appropriate safety mitigation
strategies. These two percentages are not enough to conclude that pilots are less
error-prone than mechanics because their environments are different; the difference might
be due to their management support. However, the researcher has concluded that
Brazilian Naval Aviation should direct its efforts to the maintenance operations in order
to reduce this type of human factors related incidents.
76
The most common chain of factors, totaling 10 occurrences, put together Skillbased Error and Inadequate Supervision, with gaps in HFACS Levels 2 and 4. This
result reflects the SIPAAerM belief that no active failure occurs without an associated
supervision failure (Marinha do Brasil, 2002), which is considered a good approach in
reducing the pressure on the operator. Making the supervisor accountable for the
operator’s work is a proper way to share responsibility and guarantee supervisor’s
involvement. However, simply stating that there is inadequate supervision does not help
in mitigating the problem. One of the strengths of the HFACS framework is the
definition of four different types of unsafe supervision, each with a different mitigation
strategy.
Since the incident investigators did not have the HFACS framework knowledge
during their investigations, the researcher was only able to code the other three types of
unsafe supervision when enough information was provided in the incident report. Not
surprisingly, the most common factor in HFACS Level 3 was Inadequate Supervision
totaling more occurrences that the other three unsafe supervision factors combined.
Moreover, the second most common occurrence in this level was NIL, indicating that
there was not enough information to code an unsafe supervision factor. Planned
Inappropriate Operations was the third most common factors in this HFACS level,
which might be interpreted as the existence of deficiencies in operations planning
throughout the organization. Failed to Correct Problem and Supervisory Violations
presented the same percentage, less than 10%, but the results of these last three HFACS
factors must be carefully analyzed due to the lack of investigative information.
77
The second most common chain of factors was Skill-based Error and Adverse
Mental State, with nine occurrences, followed by Skill-based Error and CRM, with eight
occurrences. In both cases, there was no code applied to HFACS Levels 3 and 4. These
results might be interpreted as result of the introduction of the Aviation Psychologists
work in the investigatory process. These professionals were responsible for the
psychological life of all personnel working within a squadron. Therefore, in the case of
an incident, the psychologist should be involved in the human factors part of
investigation (Marinha do Brasil, 2005a). Both CRM and Adverse Mental State factors
would have been easy to detect by human factors specialist, even more so when the
specialist already knows the person involved in the unsafe act.
Of the five most common chains of factors, four did not contain codes on Levels 3
and 4. This result might represent a superficial investigation of the supervisory and
organizational levels of the incident or a misunderstanding of the influence of these levels
as latent failures leading to unsafe acts. In the first case, it is important to remember that
these investigations were conducted in a military organization, and the Commanding
Officer of each squadron has to approve any investigation conducted within the squadron
(Marinha do Brasil, 2005a). This procedure might cause some bias or superficial
investigation due to fear of reprisal when problems are uncovered during the incident
investigation. In the second case, the investigator might limit the depth of the search for
latent failures due to limited comprehension of all the influences that can play a role in
accidents. Without an appropriate framework to guide the investigative process, even the
best investigator might have problems uncovering the latent failures in higher levels,
78
especially when the investigator works within the organization and shares the same safety
culture.
The first HFACS Level 4 factor to appear in the list of most common chain of
factors was Organizational Process, tied with other seven chains of factors with four
occurrences. Moreover, HFACS Level 4 factors appear only four times in the list of the
25 more common chains of factors in this study, and the absence of Level 4 factors
occurred in two thirds of the chains of factors coded in this study. These results agree
with and enforce the discussion in last paragraph in regards to the lack of an appropriate
framework to guide the investigative process to uncover human factors associated with
incidents.
The most common factor in this HFACS level was Organizational Process, with
16.4% of the cases, followed by Resource Management, with 10.7% of the cases, and
Organizational Climate, with 6.2%. Any discussion of these results should take into
consideration the limited data available in this study. However, these results could be
used to prioritize safety mitigation strategies within Brazilian Naval Aviation. The initial
focus should be on the organizational processes implemented by rules and regulations.
This approach is important because this HFACS Level is the one in which mitigation
strategies can affect a greater number of people and thus be more effective in
implementing changes organization-wide. The lack of investigative information at this
level might hinder a safety approach capable of solving many problems in lower HFACS
levels, but these results provide an idea of where to prioritize the application of resources
in developing mitigation strategies.
79
The first time Violation was included in the list of 25 most common chains of
factors, it tied with other seven chains of factors, with four occurrences. Its chain of
factors included Adverse Mental State and Inadequate Supervision, but no factor in
HFACS Level 4. This result led the researcher to wonder if supervisory failures highly
influence the way operators think and behave, leading to violations. If the violations
were condoned by the organization, they would be classified as routine violations.
However, the limited data in the summaries were not sufficient to determine the type of
violations, and further analysis of the investigative process is necessary. Were the
HFACS framework used to investigate these cases, the investigator would be able to
determine the type of violation as well as the role the organization played in each
incident. However, because there was no HFACS factor coded in HFACS Level 4 in
these four occurrences of this chain, the researcher cannot make any conclusion regarding
organizational influence in the occurrence of violations.
Besides the analysis of the most common chain of factors, this analysis enabled
the comparison of the HFACS factors within each HFACS Level. The discussion of
HFACS Levels 3 and 4 were already included in previous paragraphs, but the first two
levels also presented relevant results.
HFACS Level 1 results did not mirror the previous research that analyzed
accidents data using HFACS (Shappell & Wiegmann, 2001; Shappell & Wiegmann,
2003; Scarborough, Bailey & Pounds, 2005; Wiegmann et al., 2005; Detwiler et al.,
2006; and Shappell et al., 2007). This difference was probably a result of the research
design applied to this study. Mixing incidents related to maintenance and flight crew
unsafe acts might have skewed the HFACS Level 1 data because the patterns of unsafe
80
acts were different between the two operations. Yet, studies analyzing flight crew unsafe
acts, Skill-based Errors, were the most frequent unsafe acts with usually more than 60%
of the cases (Shappell & Wiegmann, 2001; Shappell & Wiegmann, 2003; Scarborough,
Bailey & Pounds, 2005; Wiegmann et al., 2005; Detwiler et al., 2006; and Shappell et al.,
2007). Conversely, Decision Errors are the most frequent unsafe act in maintenance
related accidents, representing more than 50% of the cases (Krulak, 2004; Rashid &
Braithwaite, 2010). Despite the higher number of maintenance related incidents, Skillbased Error was the most common unsafe act in this study, closely followed by Decision
Errors.
The percentage of Violations and Perceptual Errors varied widely in previous
studies of flight crews unsafe acts, resulting in no discernable pattern; but this was not the
case in maintenance unsafe acts. Krulak (2004) and Rashid et al. (2010) found that
violations occurred in more than 30% of the cases. This contrasts with this study, where
flight crew violations occurred in only 0.6% of the cases and maintenance violations
occurred in 10.7% of the cases. Despite the lower number, compared to previous
research, the researcher has concluded that the difference between maintenance and flight
crew should still be addressed to decrease this type of occurrence in the maintenance
environment. As with earlier recommendations, further investigation of the violations
found in this study should be accomplished to clarify the circumstances in which they
occurred.
HFACS Level 2 presented important results along with disappointing ones. The
analysis of previous studies had not shown any specific pattern in this level because the
results were highly dependent on the data available to the researchers and mainly limited
81
by the depth of the investigation processes (Wiegmann & Shappell, 2001a). This
limitation also occurred in this study because in almost one quarter of the cases there was
no HFACS factor coded in this level, and was the second most common code applied in
HFACS Level 2. The lack of an adequate framework to guide the human factors
investigative process might have hindered the uncovering of many preconditions for
unsafe acts within the incidents analyzed in this study.
The high percentage of Adverse Mental State cases has clearly shown the
influence and efficiency of Aviation Psychologists working in each Brazilian Naval
Aviation squadron. As already mentioned, they helped the investigative process in
determining the psychological state operators were experiencing at the time of the unsafe
act. However, it seems that the investigation process were limited in their search of more
preconditions for unsafe acts. CRM was the third most common factor in this level,
totaling 15.1% of the cases. This was a surprise because CRM was introduced in
Brazilian Naval Aviation more than 20 years ago and should be used in every flight
(Marinha do Brasil, 1999). For maintenance operations, however, there has been no such
program in place. Despite the reduced number of CRM related incidents analyzed in this
study, latent failures might have been causal factors because of the absence of any type of
CRM program. Moreover, the absence of a framework that included CRM in the
maintenance environment might have hindered the analysis of communication,
coordination, and planning processes within that environment.
Physical and Mental Limitations was the fourth most common factor in this level,
with 13.8% of the cases. This result probably represented deficiencies in the training
programs within both flight crew and maintenance environment, because the selection
82
process to be accepted in the Brazilian Navy Aviation probably eliminated anyone who
did not fit the environment, either physically or mentally. Additionally, the yearlyrequired medical certification along with the work of the Aviation Psychologists should
have identified personnel with problems and prevented them from working in the aviation
environment until they were fit for duty again. Training, both initial and recurrent,
seemed to play a relevant role in creating preconditions for unsafe acts by the operator.
Another relevant result was the reduced cases of Personal Readiness and Adverse
Physiological State cases in this level, which together accounted for only 3.8% of the
cases. The first HFACS factor was related to the activities outside the work environment
that can influence the personal readiness for duty, like crew rest requirements and selfmedication. The second one dealt with acute medical or physiological conditions that
negatively affect performance of the operator. These results have shown that the
mitigation strategies already in place are achieving the results they should. Besides
numerous education programs developed by SIPAAerM, the most relevant regulation
related to this result was the aeronautical activity journey (JAA) definition established in
2003 by the PPAA and reinforced in the second edition of the Aviation Safety Manual
(Marinha do Brasil, 2003; Marinha do Brasil, 2005a).
Entire framework analysis. The analysis of the 141 incident summaries rendered
145 entire framework cases. For the four cases in which both maintenance and flight
crew played relevant roles in the incidents, two different frameworks were recorded to
permit a comparison between both environments. It was interesting to notice that the
discussion accomplished in the chain of factors analysis could also be done with the
coding procedure adopted in the entire framework analysis. The application of different
83
statistical calculations would have provided the same results, enabling the same
discussion above. The researcher has concluded that the entire framework coding
process was more reliable due to the numerous previous studies using this design, which
provided relevant results (Shappell & Wiegmann, 2001; Shappell & Wiegmann, 2003;
Scarborough, Bailey & Pounds, 2005; Wiegmann et al., 2005, Detwiler et al., 2006;
Shappell et al., 2007; Krulak, 2004; and Rashid et al., 2010).
Four out of the eighteen HFACS factors occurred in more than 40% of summaries
analyzed. Two of them were within HFACS Level 1; one was within HFACS Level 2;
one was within HFACS Level 3; none of them was within HFACS Level 4. Despite the
fact that these results might indicate a strong tendency in the occurrence of these factors,
a deeper analysis was necessary. It seems that the investigative information provided by
the summaries has not allowed other HFACS factors to be coded due to the lack of
knowledge regarding the HFACS framework during the investigative process.
Skill-based Error and Decision Error were, respectively, the first and third most
common HFACS factor coded in this study. This was not a surprise since all incidents
had at least one HFACS Level 1 factor identified. Additionally, this result has brought
together the results of both flight crew and maintenance operations, something not done
in previous studies regarding the application of HFACS. Whereas Skill-based Error was
the most common unsafe act in flight crew-related unsafe acts (Shappell & Wiegmann,
2001; Shappell & Wiegmann, 2003; Scarborough, Bailey & Pounds, 2005; Wiegmann et
al., 2005; Detwiler et al., 2006; and Shappell et al., 2007), Decision Error was the most
common unsafe act in maintenance-related unsafe acts (Krulak, 2004; Rashid et al.,
2010).
84
Adverse Mental State was the second most common HFACS factor to occur,
which might represent the relevant work of Aviation Psychologists within Brazilian
Naval Aviation. Although the percentage was relevant, the lack of others strong HFACS
factors in HFACS Level 2 might be interpreted as a positive influence from Aviation
Psychologists. However, this result might also be interpreted as a lack of deeper
investigation of the preconditions leading to unsafe acts due to investigators’ high
reliance on the Aviation Psychologists conclusions. Focusing the investigation in
psychological state of the operator might have hindered the discovery of other issues that
needed to be addressed. Inadequate Supervision was the fourth most common HFACS
factor to occur. This result could have been interpreted as a direct consequence of the
SIPAAerM position that every active failure is related to a supervisory failure (Marinha
do Brasil, 2002). The problem with this belief is that it might have hindered a deeper
investigation of supervisors’ role in the mishaps. Just stating that there was an
inadequate supervision does not help to mitigate the mishap. The use of the HFACS
framework, which provides four HFACS factors in this level, might have guided the
investigators to uncover the latent failures at this level that are hidden due to the
limitations presented by their present approach.
All other HFACS factors occurred in less than 20% of the cases. The analysis of
the relevant results of these HFACS factors was already accomplished in the chain of
factors analysis. The researcher believes that another interesting discussion would be the
analysis of the HFACS factors that presented relevant differences between maintenance
and flight crew because these HFACS factors might provide insights regarding the
differences between these two operations.
85
HFACS Level 3, Unsafe Supervision, was the level presenting the most
interesting results. Whereas Planned Inappropriate Operations occurred almost three
times more frequently in incidents related to flight crew unsafe acts, maintenance cases
occurred five times more frequently in Failed to Correct Problem, six times more
frequently in Inadequate Supervision, and 12 times more frequently in Supervisory
Violations. These results can be interpreted in two different ways. First, the
investigation process of flight crew unsafe acts did not analyze the supervisory level, the
Operations Department of each squadron, to detect latent failures that could have
contributed to unsafe acts. When an inappropriate operation has been planned and
resulted in an incident, the detection of the cause was not hard to find. However, the
other three HFACS factors in this level might need a deeper investigation. When an
Aviation Safety Officer was conducting an investigation, he might have felt pressured not
to be too intrusive in other departments, and, consequently, reduced the depth of the
investigation. This might have been a culture deficiency within Brazilian Naval
Aviation, because the final goal of any investigation should be to prevent the mishap
from happening again, not to impose blame on someone or show failures in someone’s
work. The investigation process should be seen as an opportunity for learning and
improving to prevent future mishaps.
Second, the supervisory level of maintenance operations seems to be the weakest
point within that environment. The SIPAAerM policy regarding the supervisory
accountability seems to be appropriate in discovering the latent failures, but more should
be done in order to reduce its occurrence. Simply stating that there is a supervision
deficiency does not do any good in developing the appropriate mitigation strategy. The
86
use of a framework like HFACS may guide the investigative process and allow the proper
classification of the latent failures at this level as well as the development of more
effective safety recommendations is warranted. This is important because the mitigation
strategies to each HFACS factor in the supervisory level are remarkably different. For
example, inadequate supervision cases might involve the necessity of more training for
operators, whereas supervisory violation cases might involve the enforcement of already
established rules.
HFACS Level 4, Organizational Influences, also presented relevant results.
Whereas Organizational Climate had similar percentages for flight crew and maintenance
environment, Resource Management and Organizational Process occurred more
frequently in maintenance-related incidents. These results enforced the aforementioned
need to improve safety mitigation approaches in the maintenance environment as well as
providing the operators with a proper environment in which they are not led to unsafe
acts due to improper procedures, misplaced policy, or misuse of scarce resources. The
organizational culture might also influence this environment, but the results showed that
this influence was the same to maintenance and flight crew operations.
Another important result was the large difference in the percentage of CRM
occurrences between flight crew and maintenance incidents. Flight crew incidents
occurred more than seven times more frequently than maintenance incidents. This
difference might have reflected a misconception that CRM is limited to the flight crew
environment. Until today, Brazilian Naval Aviation had not implemented the concept of
Maintenance Resource Management (MRM), created from the basic concepts of CRM,
but applied to the maintenance environment. MRM is a two-fold endeavor to improve
87
labor-management cooperation for safety and to develop positive assertiveness
communications practices (FAA, 2000). Reason’s (1990) Swiss Cheese theory provides
the basis for MRM philosophy along with safety culture considerations (FAA, 2000).
Since MRM involves training in communications skills, teamwork skills, situation
awareness, and leadership; it also has the potential to reduce the relevance of latent
failures in the supervisory level because those supervisors would be more trained to cope
with the situations they are exposed to in a daily basis. In a study, Patankar and Taylor
(2008) concluded that the implementation of MRM decreased the occurrence of ground
damage, logbook errors, and increased the awareness of human performance issues
within the organizations.
The introduction of MRM training might also be able to solve the violation
problems encountered in the maintenance operations within Brazilian Naval Aviation.
When compared to flight crew, maintenance operations presented violation 15 times
more frequently. MRM introduces the concept of norm, which refers to the unwritten
procedures accepted by the organization culture, despite the fact that they are a violation
of the formally established ones (FAA, 2000). Within the HFACS framework, this
violation would be considered a routine violation because it was condoned by supervisory
or management levels. Norms should be prevented because they set the stage for other
violations (FAA, 2000).
Even though SIPAAerM have introduced the distinction between human error and
violation in 2005 (Marinha do Brasil, 2005), it seems that investigators are still reluctant
to classify an unsafe act as a violation probably due to fear of reprisal to the investigator
or to the operator. However, according to the HFACS framework, not all types of
88
violations are subject to reprisal or disciplinary action. If an operator violates a rule
because he was following guidance from his supervisor, he cannot be made accountable.
Even more, if that practice was known and condoned by management, the organization
has a larger influence in violations than any single worker does. The researcher
concluded that the introduction of MRM, along with HFACS, would probably help
decrease occurrence of violations due to the improved awareness of the factors that might
lead to them.
Reliability testing. The intra-rater reliabilities were calculated in this study for
each HFACS factor and for the complete coding process by using Cohen’s Kappa
coefficient. The values obtained ranged between 0.46 and 1.00, spanning between
moderate agreement and almost perfect agreement. Six HFACS factors exceeded a
Kappa of 0.80, which indicated almost perfect substantial agreement; nine HFACS
factors had Kappa values between 0.60 and 0.79 indicating substantial levels of
agreement; three HFACS factors had Kappa values between 0.40 and 0.59 indicating
moderate levels of agreement. Intra-rater reliabilities calculated as a simple percentage
rate of agreement obtained reliability figures between 79.3% and 100%, also indicating
acceptable reliability between the rounds of coding process.
The indexes of reliability using Cohen’s Kappa and percentage of agreement
between raters were discrepant in only two HFACS factors. Failed to Correct Problem
presented the lowest Cohen’s Kappa coefficient, 0.46, but the percentage agreement was
91.7%, which is significantly higher than the lowest percentage agreement of 79.3%.
Resource Management presented a Cohen’s Kappa coefficient of 0.58 and a percentage
agreement of 92.4%, falling into the same situation. The explanation for this is issue
89
might be the low number of occurrences of these HFACS factors, 12 and 16,
respectively. These low frequencies might be unreliable and might easily distort the
Cohen’s Kappa value in such instances, actually deflating its value where there was
actually a very high level of agreement (Li & Harris, 2005).
In a general comparison among the HFACS levels, HFACS Level 1, Unsafe Acts,
showed the lowest Cohen’s Kappa coefficient values. This might be a result of the expert
reliability check conducted by the researcher with two HFACS experts. After analyzing
and discussing a sample of two incidents, the researcher had to reanalyze some incidents
to follow further instructions and orientations provided by the experts. The focus of this
reanalysis was the recoding of the unsafe acts. The changes were made on the results of
second round of coding process. HFACS Level 2, Preconditions for Unsafe Acts,
presented the highest Cohen’s Kappa coefficients, with five out of seven HFACS factors
showing almost perfect agreement levels. HFACS Levels 3 and 4 did not present any
relevant characteristics in the Cohen’s Kappa values.
Wiegmann and Shappell (2001b) found that HFACS framework as a whole had a
Cohen’s Kappa coefficient of 0.71 for inter-rater reliability, which indicated substantial
agreement. This study found a Cohen’s Kappa coefficient of 0.75 for the whole
framework. This result can be interpreted as strength of this study, despite the limitation
of the researcher being the only person coding the data.
Conclusions
The analysis of Brazilian Naval Aviation incidents using the HFACS framework
provided interesting insights addressing the underlying conditions that contributed to
human error. It not only identified human’s unsafe acts, but also provided a better view
90
of the latent risks that lead to those unsafe acts. This study design made use of two
different coding processes in order to answer the two research questions, but the chain of
factors analysis was deemed unnecessary due to its limitations and to the
misinterpretation that can result if the reader is not trained in the HFACS framework. All
of the results achieved through the chain of factors analysis could have been achieved by
the entire framework analysis.
The incidents’ demographic analysis presented expected results. Aircraft that are
more complex had more human error-related incidents. The human factors approach
improvements achieved with the introduction of Aviation Psychologist around 10 years
ago led to a remarkable reduction in the number of human error-related incidents; SBES
was the location in which the largest number of incidents occurred due to the
concentration of operations there; however, operations at other airports presented a higher
percentage of human errors than operations at ships, which are a more challenging
environment for both flight and maintenance crews. Complacency might occur when
operating at airports, but a deeper investigation is needed to identify the reasons
associated with this issue as well as to establish mitigation strategies.
Skill-based Errors and Decision Errors were identified as the most common
Unsafe Act in the incidents analysis, which is in agreement with the literature and
previous studies applying HFACS. Adverse Mental State was the most common
Precondition for Unsafe Act factor, representing the relevance of the Aviation
Psychologist’s work within each squadron in Brazilian Naval Aviation. Inadequate
Supervision was the most common Unsafe Supervision factor, showing that investigators
strictly follow SIPAAerM position that for every active failure, there is an associated
91
supervisory failure. Nonetheless, limiting the investigation to identify only an inadequate
supervision might hinder a broader approach to the supervisors’ role in the incidents.
Despite the reduced number of factors coded in the Organizational Influence level,
Organizational Process was the most common factor identified in this analysis,
presenting the area in which the Brazilian Naval Aviation should initially focus its
resources on new mitigation strategies.
Other HFACS factors also presented relevant results that can guide safety
mitigation strategies in the future. The introduction of JAA and the numerous
educational programs regarding the necessity of operators being in a good physical state
in order to develop high-level tasks reflected a reduced number of Personal Readiness
and Adverse Physiological State cases. Violations occurred remarkably less often than in
previous studies, but the difference in occurrence between flight crew and maintenance
operations was deemed to indicate that maintenance operations need a systematic
approach in order to reduce this occurrence. MRM, as a CRM program applied to
maintenance operations, seems to be the best initial available approach that might also
mitigate other issues such as Supervisory Violations and Planned Inappropriate
Operations. Moreover, maintenance operations were related to 60% of the incidents
analyzed in this study. This result warrants a need for further investigation in the
maintenance operations environment in order to uncover the issues not yet addressed by
safety mitigation strategies. The use of MRM might facilitate this investigation as well.
The CRM, as an HFACS factor, results implied that communications within the cockpit
might still present some barriers, despite the introduction of CRM philosophy in
Brazilian Naval Aviation more than 20 years ago.
92
As the investigative processes and rules evolved within Brazilian Naval Aviation,
more data regarding to human errors has been made available in incident summaries
providing relevant information to allow for the use of the HFACS framework. However,
there is always room for improvement. The issuance of one summary for two or more
incidents hindered the coding process of some incident summaries due to the lack of
investigative information provided. Therefore, for future studies, each incident should
have its own associated summary. HFACS Level 4 had very limited information in the
incident summaries, which might reflect lack of an appropriate framework to guide the
human factors investigation process or fear of reprisal from the organization. The
implementation of the HFACS framework as a formal tool in investigations might solve
both problems because the fear of reprisal might diminish with the use of a SIPAAerM
approved framework. The unique caveat is that the introduction of the HFACS
framework should be done with the appropriate training to allow the correct collection of
data and avoid misinterpretation of the framework.
The results presented in this study should be enough to elicit a deeper study into
the underlying factors related to human errors within Brazilian Naval Aviation; however,
some recommendations might optimize the application of limited resources available.
SIPAAerM, as the Aviation Safety organization within Brazilian Naval Aviation, should
be the focus of all of those recommendations.
SIPAAerM should keep enforcing all present safety regulations and approaches,
because none of them was deemed inappropriate in this study finding. Nonetheless, there
are some fixes to be implemented. SIPAAerM should avoid issuing one incident
summary for more than one incident to prevent problems in future research using the data
93
reported in its summaries. SIPAAerM should also evaluate the introduction of the
HFACS framework as a tool to all investigations within Brazilian Naval Aviation. This
introduction should be planned following an appropriate training schedule for all
Aviation Psychologists and Aviation Safety Officers (OSAv) along with the introduction
of the framework to all personnel involved in aviation operations. The involvement of all
personnel will preclude problems during future investigations in which the HFACS
framework might be in use. Moreover, SIPAAerM should evaluate the introduction of a
formal MRM program within Brazilian Naval Aviation to address the issues discovered
in this study in regards to the maintenance operations, mainly violations and supervisory
issues. Finally, a MRM implementation might also help the introduction of the HFACS
framework because it would improve communications and coordination environment in
the maintenance operations once MRM programs are in place.
Recommendations
During the introduction of the HFACS within Brazilian Naval Aviation,
SIPAAerM should create two research projects to support HFACS implementation. The
first project should establish the training of at least two Aviation Psychologists in the
HFACS framework in order to allow them to code the same incidents used in this study.
A similar agreement procedure should be adopted as done in this study to obtain the final
codes and an agreement level within all three coders. This research project should
provide an answer to the limitations imposed on the present study.
The second project should establish a team of Aviation Psychologists to analyze
the Brazilian Naval Aviation accident reports that were excluded from this study due to
different investigation procedures. This research project should determine the HFACS
94
factors related to the accidents, and enable a comparison with the first research project.
The relevance of this research project relies on the fact that deficiencies in both
investigatory processes might be discovered in this comparison; thus, enabling changes
that can enhance the aviation safety processes and regulations within Brazilian Naval
Aviation.
95
References
Australian Transport Safety Bureau. (2008). Evaluation of the human factor analysis and
Classification system as a predictive model. Canberra City, Australia: Australian
Transport Safety Bureau.
Baker, S. (1999). Aviation incident and accident investigation. In D. J. Garland, J. A.
Wise & V. D. Hopkin (Eds.), Handbook of aviation human factors (pp. 631-641).
Mahwah, NJ: Lawrence Erlbaum.
Beaubien, J. M. & Baker, D. P. (2002). A review of selected aviation human factors
taxonomies, accident/incident reporting systems, and data reporting tools.
International Journal of Applied Aviation Studies, 2(2), 11-36.
Boeing Company. (n.d.). Maintenance error decision aid (MEDA) users guide. Retrieved
from the FAA website: http://www.hf.faa.gov/hfguide/07/07_MEDA_
Users_Guide.doc
Comando da Aeronáutica. (2008a). MCA 3-1 Conceituação de vocábulos, expressões e
siglas de uso do SIPAER [Definition of terms, expressions and abbreviation used
by SIPAER]. Brasília, Brazil. Retrieved from CENIPA Website:
http://www.cenipa.aer.mil.br/cenipa/ paginas/normas/NSCA3-1.pdf
Comando da Aeronáutica. (2008b). MCA 3-2 Investigação e prevenção de acidentes
aeronáuticos [Aeronautical Accidents Investigation and Prevention]. Brasília,
Brazil. Retrieved from CENIPA Website: http://www.cenipa.aer.mil.br/cenipa/
paginas/normas/NSCA3-2.pdf
Comando da Aeronáutica. (2008c). MCA 3-6 Manual de investigação do SIPAER
[SIPAER Investigation Manual]. Brasília, Brazil. Retrieved from CENIPA
Website: http://www.cenipa.aer.mil.br/cenipa/paginas/normas/NSCA3-6.pdf
Dambier, M., & Hinkelbein, J. (2006). Analysis of 2004 German general aviation aircraft
accidents according to the HFACS model. Air Medical Journal, 25(6), 265-269.
Dekker, S. (2006). The field guide to understand human error. Hampshire, England:
Ashgate.
Detwiler, C., Hackworth, C., Holcomb, K., Boquet, A., Pfleiderer, E., Wiegmann, D. A.,
& Shappell, S. A., (2006). Beneath the tip of the iceberg: A human factors
analysis of general aviation accidents in Alaska versus the rest of the United
States (DOT/FAA/AM-06/7). Washington, DC: Federal Aviation Administration,
Office of Aerospace Medicine.
96
Federal Aviation Administration. (2000). Maintenance Resource Management Training
(Advisory Circular 120-72). Retrieved from http://rgl.faa.gov/Regulatory_
and_Guidance _Library/rgAdvisoryCircular.nsf/list/AC%20120-72/$FILE/
AC%20120-72.pdf
Federal Aviation Administration. (2008). Human factors guide for aviation maintenance
and inspection. Retrieved from http://www.hf.faa.gov/hfguide/index.html
Fleiss, J., Levin, B., & Paik, M. C. (2003). Statistical methods for raters and proportions
(3rd Edition). New York: Wiley, John & Sons, Inc.
Gaur, D. (2005). Human factor analysis and classification system applied to civil aircraft
accident in India. Aviation, Space, Environment Medicine, 76(5), 501-505.
Hollnagel, E. (2004). Barriers and accident prevention. Aldershot, England: Ashgate.
International Civil Aviation Organization. (1976). Manual of aircraft accident
investigation. Montreal, Canada: Author.
International Civil Aviation Organization. (1993). Human factors digest No. 7:
Investigation of human factors in accidents and incidents (Circular 240-AN/14).
Montreal, Canada: Author.
International Civil Aviation Organization. (2006). Annex 13 to the convention on
international civil aviation: Aircraft accident and incident investigation.
Montreal, Canada: International Civil Aviation Organization Documents Sales.
Isaac, S., & Michael, W. B. (1995). Handbook in research and evaluation. San Diego,
CA: Educational and Industrial Testing Services.
Krulak, D. C. (2004). Human factors in maintenance: Impact on aircraft mishap
frequency and severity. Aviation, Space, and Environment Medicine, 75(5), 429432.
Lenné, M. G., Ashby, K., & Fitzharris, M. (2008). Analysis of general aviation crashes in
Australia using the human factors analysis and classification system. The
International Journal of Aviation Psychology, 18(4), 340-352.
Li, W., & Harris, D. (2005). HFACS analysis of ROC Air Force aviation accidents:
reliability analysis and cross-cultural comparison. International Journal of
Applied Aviation Studies, 5(1), 65-81.
Li, W., & Harris, D. (2006). Pilot error and its relationship with higher organizational
levels: HFACS analysis of 523 accidents. Aviation, Space, and Environment
Medicine, 77(10), 1056-1061.
97
Li, W., Harris, D., & Yu, C. (2008). Routes to failure: Analysis of 41 civil aviation
accidents from the Republic of China using the human factors analysis and
classification system. Accident Analysis and Prevention, 40(2), 426-434.
Majumdar, A., Mak, K., Lettington, C., & Nalder, P. (2009). A causal factor analysis of
helicopter accidents in New Zealand 1996-2005 and the United Kingdom 19862005. The Aeronautical Journal, 113(1148), 647-660.
Marinha do Brasil. (1999). Programa de prevenção de acidentes aeronáuticos de 1999.
[1999 Aeronautical Accidents Prevention Program]. Rio de Janeiro, Brazil:
Diretoria de Aeronáutica da Marinha [Brazilian Navy Directorate of Aeronautics].
Marinha do Brasil. (2000). Manual de segurança de aviação [Aviation Safety Manual]
(1st ed.). Rio de Janeiro, Brazil: Diretoria-Geral de Material da Marinha
[Brazilian Navy Material General-Directorate].
Marinha do Brasil. (2002). Programa de prevenção de acidentes aeronáuticos de 2002.
[2002 Aeronautical Accidents Prevention Program]. Rio de Janeiro, Brazil:
Diretoria de Aeronáutica da Marinha [Brazilian Navy Directorate of Aeronautics].
Marinha do Brasil. (2003). Programa de prevenção de acidentes aeronáuticos de 2003.
[2003 Aeronautical Accidents Prevention Program]. Rio de Janeiro, Brazil:
Diretoria de Aeronáutica da Marinha [Brazilian Navy Directorate of Aeronautics].
Marinha do Brasil. (2005a). Manual de segurança de aviação [Aviation Safety Manual]
(2nd ed.). Rio de Janeiro, Brazil: Diretoria-Geral de Material da Marinha
[Brazilian Navy Material General-Directorate].
Marinha do Brasil. (2005b). Programa de prevenção de acidentes aeronáuticos de 2005.
[2005 Aeronautical Accidents Prevention Program]. Rio de Janeiro, Brazil:
Diretoria de Aeronáutica da Marinha [Brazilian Navy Directorate of Aeronautics].
Marinha do Brasil. (2008). Programa de prevenção de acidentes aeronáuticos de 2008.
[2008 Aeronautical Accidents Prevention Program]. Rio de Janeiro, Brazil:
Diretoria de Aeronáutica da Marinha [Brazilian Navy Directorate of Aeronautics].
Marinha do Brasil. (2009). Programa de prevenção de acidentes aeronáuticos de 2009.
[2009 Aeronautical Accidents Prevention Program]. Rio de Janeiro, Brasil:
Diretoria de Aeronáutica da Marinha [Brazilian Navy Directorate of Aeronautics].
Marinha do Brasil. (2010). Programa de prevenção de acidentes aeronáuticos de 2010.
[2010 Aeronautical Accidents Prevention Program]. Rio de Janeiro, Brazil:
Diretoria de Aeronáutica da Marinha [Brazilian Navy Directorate of Aeronautics].
98
McVenes, T., & Chidester, T. R. (2005). Changing national safety culture through data
sharing. Proceedings of the 58th International Air Safety Seminar (IASS),
Moscow, Russia, 303-315.
Norman, D. A. (1988). The psychology of everyday things. New York, NY: Basic Books.
Patankar, M. S., & Taylor, J. C. (2008). MRM training, evaluation, and safety
management. The International Journal of Aviation Psychology, 18(1), 61-71.
Portney, L. G., & Watkins, M. P. (2000). Reliability. In. L.G. Portney and M.P. Watkins
(Eds) Foundations of clinical research: applications to practice (pp. 61-77). NJ:
Prentice Hall.
Rashid, H. S. J., Place, C. S., & Braithwaite, G. R. (2010). Helicopter maintenance error
analysis: beyond the third order of the HFACS-ME. International Journal of
Industrial Ergonomics, 40, 636-647.
Reason, J. (1990). Human error. New York, NY: Cambridge University Press.
Reason, J. (1997). Managing the risks of organizational accidents. Aldershot, England:
Ashgate.
Scarborough, A., Bailey, L., & Pounds, J. (2005). Examining ATC operational errors
using the human factors analysis and classification system (DOT/FAA/AM05/25). Washington, DC: Federal Aviation Administration, Office of Aerospace
Medicine.
Schmorrow, D. D. (1998). A human error analysis and model of naval aviation
maintenance related mishaps (Master’s Thesis). Retrieved from
http://handle.dtic.mil/100.2/ADA355995
Shappell, S. A., Detwiler, C., Holcomb, K., Hackworth, C., Boquet, A., & Wiegmann, D.
A. (2007). Human factors and commercial aviation accidents: An analysis using
the human factor analysis and classification system. Human Factors, 49(2), 227242.
Shappell, S. A., & Wiegmann, D. A. (2000). The human factors analysis and
classification system – HFACS (DOT/FAA/AM-00/7). Washington, DC: Federal
Aviation Administration, Office of Aerospace Medicine.
Shappell, S. A., & Wiegmann, D. A. (2001). Applying Reason: The human factor
analysis and classification system. Human Factors and Aerospace Safety, 1(1),
59-86.
99
Shappell, S. A., & Wiegmann, D. A. (2003). A human error analysis of general aviation
controlled flight into terrain accidents occurring between 1990-1998
(DOT/FAA/AM-03/4). Washington, DC: Federal Aviation Administration, Office
of Aerospace Medicine.
Strauch, B. (2002). Investigating human error: incidents, accidents, and complex
systems. Aldershot, England: Ashgate.
Tvaryanas, A. P., Thompson, W. T., & Constable, S. H. (2006). Human factors in
remotely piloted aircraft: HFACS analysis of 221 mishaps over 10 years.
Aviation, Space, and Environment Medicine, 77(7), 724-732.
United States Naval Safety Center – USNST. (2001). Human factors analysis and
classification system (HFACS): A human error approach to accident
investigation, OPNAV 3750.6R (Appendix O). Retrieved from
www.safetycenter.navy.mil/aviation/aeromedical/download/HFACSPrimer.doc
Viera, A. J., & Garret, J. M. (2005). Understanding interobserver agreement: the kappa
statistic. Family Medicine, 37(5), 360-363.
Wiegmann, D. A., Faaborg, T., Boquet, A., Detwiler, C., Holcomb, K., & Shappell, S. A.
(2005). Human error and general aviation accidents: A comprehensive, finegrained analysis using HFACS (DOT/FAA/AM-05/24). Washington, DC: Federal
Aviation Administration, Office of Aerospace Medicine.
Wiegmann, D. A., & Shappell, S. A. (1997). Human factors analysis of post-accident
data: Applying theoretical taxonomies of human error. The International Journal
of Aviation Psychology, 7(1), 67-81.
Wiegmann, D. A., & Shappell, S. A. (2001a). A human error analysis of commercial
aviation accidents using the human factors analysis and classification system
(HFACS) (DOT/FAA/AM-01/3). Washington, DC: Federal Aviation
Administration, Office of Aviation Medicine.
Wiegmann, D.A. & Shappell, S.A. (2001b). Human error analysis of commercial
aviation accidents: application of the human factors analysis and classification
system. Aviation, Space, and Environmental Medicine, 72 (11),1006-1016.
Wiegmann, D. A., & Shappell, S. A. (2003). A human error approach to aviation
accident analysis: The human factors analysis and classification system.
Aldershot, Great Britain: Ashgate.
100
Appendix A
Bibliography
101
American Psychological Association. (2010). Publication manual of the American
Psychological Association (6th edition). Washington, DC: Author.
Fleiss, J. L., Levin, B., & Paik, M.C. (2003). Statistical methods for rates and
proportions (3rd ed.). Hoboken, NJ: John Wiley & Sons, Inc.
Gay, L. R., & Airasian, P. (2000). Educational research: Competencies for
analysis and applications (6th ed.). Upper Saddle River, NJ: Merrill Prentice
Hall.
Gay, L. R., Mills, G. E., & Airasian, P. (2008). Educational research: Competencies for
analysis and applications (9th ed.). Upper Saddle River, NJ: Merrill Prentice
Hall.
Vogt, W. P. (2007). Quantitative research methods for professionals. Boston, MA: Allyn
and Bacon.
102
Appendix B
Permission to Conduct Research
103
104
Appendix C
HFACS Framework
105
HFACS framework divided into its four levels
106
Appendix D
Sample Data of Chain of Factors Analysis
107
Ordinal Number
1
2
3
05/99
01/02
05/06
AI
GO
AI
EsqdVF-1
EsqdHU-2
EsqdHU-1
AF-1A
UH-14
UH-13
Summary Number
Incident Type
Operator
Model
Year
1999
2002
2006
Location
SBES
SNJM
SNJM
Flight Phase
Take-off
Landing Pattern
Take-off
Costs
$1,700.00
$1,920.00
$1,920.00
Type
Maintenance
Maintenance
Flight Crew
Level 1 Factor
Skill-Based Error
Skill-Based Error
Decision Based Error
Level 2 Factor
Crew Resource Management
Technological Environment
Adverse Mental State
Level 3 Factor
Inadequate Supervision
Inadequate Supervision
Supervisory Violations
Level 4 Factor
Resource Management
Organizational Climate
Organizational Process
The data contained in this sample does not reflect any specific incident analyzed. The
researcher randomly changed some fields in order to prevent the disclosure of
confidential information.
108
Appendix E
Sample Data of Entire Framework Analysis
109
Ordinal Number
Summary Number
Incident Type
Operator
Model
1
2
3
05/99
01/02
05/06
AI
GO
AI
EsqdVF-1
EsqdHU-2
EsqdHU-1
AF-1A
UH-14
UH-13
Year
1999
2002
2006
Location
SBES
SNJM
SNJM
Flight Phase
Take-off
Take-off
Take-off
Costs
$1,700.00
$1,920.00
$1,920.00
Type
Maintenance
Flight Crew
Flight Crew
Skill-Based Error
1
1
0
Perceptual Error
0
0
0
Decision Error
0
0
1
Violation
0
0
0
Physical Environment
0
0
0
Technological Environment
0
1
0
Adverse Mental State
0
0
1
Adverse Physiological State
0
0
0
Physical/Mental Limitations
0
0
0
Crew Resource Management
1
0
1
Personal Readiness
0
0
0
Inadequate Supervision
1
1
0
Planned Inappropriate Operations
0
0
0
Failure to Correct Problem
0
0
0
Supervisory Violations
0
0
1
Resource Management
1
0
0
Organizational Climate
0
1
0
Organizational Process
0
0
1
The data contained in this sample does not reflect any specific incident analyzed. The
researcher randomly changed some fields in order to prevent the disclosure of
confidential information.