How to Audit
the Difficult Areas
of a
Quality Management
System
Whittington & Associates, LLC
242 Highlands Drive, Woodstock, GA 30188
www.WhittingtonAssociates.com
800-404-7585 or 770-517-7944
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 1
Introduction
Some parts of a quality management system are
more difficult for auditors to assess:
1. Undocumented Process
2. Legal Requirements
3. Resource Management
4. Continual Improvement
5. Preventive Action
6. Internal Audits
7. Process Effectiveness
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 2
Introduction
To discuss how to best audit these areas, we first
have to clearly understand the requirements.
Then, we need to remember that auditors collect
evidence from these primary sources:
• Interviews (statements from responsible persons)
• Observations (demonstrations and operations)
• Documents (plans, procedures, and instructions)
• Records (past practices as proof of conformity)
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 3
1. Undocumented Process
• Documents required by ISO 9001 (per 4.2.1.a-c)
–
–
–
–
Quality Policy; Quality Objectives; Quality Manual
Document Control and Record Control Procedures
Internal Audit and Nonconformity Control Procedures
Corrective Action and Preventive Action Procedures
• And, documents needed for effective planning,
operation, and control of processes (per 4.2.1.d)
• Work instructions are optional (unless operating
under industry sector scheme like ISO/TS 16949)
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 4
Undocumented Process
•
•
•
•
•
•
•
•
•
How audit if requirements aren’t documented?
Ask the process owner to describe the process
Use manager statement as requirement source
Carefully watch the process being performed
See if documents actually exist at work place
Examine records to match practices to intent
Write nonconformity report if find a discrepancy
Action doesn’t have to include adding document
Avoid suggesting expanded text just for auditor
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 5
2. Legal Requirements
Does ISO 9001 address legal requirements? Yes.
• 5.1.a - Top management must communicate
importance of meeting customer, as well as,
statutory and regulatory requirements
• 7.2.1.c - Organization must determine statutory
and regulatory requirements for product
• 7.3.2.b - Inputs to design must include applicable
statutory and regulatory requirements
These legal requirements are for quality system
and product, not health, safety, or environment.
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 6
Legal Requirements
•
•
•
•
•
•
•
•
Identify applicable legal requirements for area
Ask legal staff, contract group, and audited area
Ensure requirements are available for reference
See if monitor for new or changed requirements
Request evidence of conformity to requirements
Issue NC if legal requirements not considered
Issue NC if area in violation of legal requirement
Help area to comply with statutes and regulations
Requirements: customer, company, standard, legal
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 7
3. Resource Management
• ISO 9001, clause 6.1, requires organization to
determine and provide resources needed to:
– Implement and maintain quality system
– Continually improve system effectiveness
– Enhance customer satisfaction
(by meeting customer requirements)
• Resources include: equipment, facilities, people,
supporting services, work environment, suppliers,
information, natural resources, and finances
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 8
Resource Management
• Are resources being identified, planned, made
available, used, monitored, and changed?
• Assessing performance to evaluate resources?
• Don’t audit in isolation; verify performance results
• Interview top management; examine the evidence
• Don’t make subjective judgments on adequacy
• Limit role to judging effectiveness of resources
• Avoid being placed in middle of resource dispute
• Issue NC on “problem” due to lack of resources
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 9
4. Continual Improvement
Continual Improvement is the “recurring activity to
increase the ability to fulfill requirements.”
Clause 8.5.1 requires continual improvement of the
effectiveness of QMS by use of quality policy, quality
objectives, audit results, data analysis, corrective
action, preventive action, and management review.
• Effectiveness is “extent to which planned activities
are realized and planned results achieved.”
• Quality Policy, 5.3, must include a commitment to
continual improvement of effectiveness of QMS
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 10
Continual Improvement
• Are continual improvement projects identified?
(beyond corrective and preventive actions)
• How were rates of improvement determined?
• Are plans approved and resources allocated?
• Keyed to requirements and satisfying customers?
• Compare performance results to quality targets
• Not a nonconformity if targets are not being met
• If not met, analyzing why and revising the plan?
• Unable to improve in all areas at once (prioritize)
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 11
5. Preventive Action
“The action to eliminate the cause of a potential
nonconformity or other undesirable situation.”
• ISO 9001 requires documented PA procedure
• Combined CA and PA procedure is acceptable
• Determine action to eliminate causes of potential
nonconformities to prevent their occurrence
• Action must be appropriate to effects of problem
• Evaluate need; determine and implement action
• Keep records of results; review actions taken
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 12
Preventive Action
•
•
•
•
•
•
•
•
Understand PA versus Correction versus CA
How are potential nonconformities identified?
Best time is early in product cycle, e.g., FMEA
Look at the nonconformity trends and patterns
Examining warning signals for out-of-control?
Look at records of preventive actions and results
Verify action effectively prevented potential NC
Goal of PA is avoiding possible NC (status quo)
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 13
6. Internal Audits
Audit: a systematic, independent, and documented
process for obtaining audit evidence and evaluating
it objectively to determine the extent to which audit
criteria are fulfilled.
Conducted at planned intervals to determine if the
quality management system conforms to:
– Planned arrangements
– ISO 9001 requirements
– Organization requirements
and is “effectively” implemented and maintained.
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 14
Internal Audits
Describe audit process in documented procedure.
Plan the audit program to consider:
– Status and importance of processes and areas
– Results of previous audits
Define criteria, scope, frequency, and methods.
Select auditors, and conduct audits, to ensure:
– Objectivity
– Impartiality
Do not audit your own work.
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 15
Internal Audits
•
•
•
•
•
•
•
•
•
Are scheduled audits conducted as planned?
Are all functional areas and shifts being audited?
Are the auditors competent and independent?
Do audit reports show procedure being followed?
Is schedule adjusted based on past audit results?
Is more audit attention given to high risk areas?
Do audits examine conformity and effectiveness?
Are all requirement types used as audit criteria?
Are audits conducted using “process approach”?
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 16
Internal Audits
• Are weaknesses in poorly performing processes
being identified by audits?
• Are NCs spotted before found in external audits?
• Are OIs being identified by internal auditors?
• Are CAs properly verified before audit closure?
• Are audit program objectives set, tracked, met?
• What is auditee and management feedback?
• Have any OIs been identified for audit process?
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 17
7. Process Effectiveness
Audit focus usually on conformity, not effectiveness.
Requirement is to audit effectiveness of processes.
Process is a set of interrelated or interacting
activities which transform inputs into outputs.
Process Approach is the systematic identification
and management of processes, and particularly
their interactions.
Effectiveness = extent to which planned activities
are realized and planned results achieved.
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 18
Turtle Diagram
Resources
Resources
What?
Who?
R
E
Q
U
I
R
E
M
E
N
T
S
R
E
Q
U
I
R
E
M
E
N
T
S
INPUT
Receive
What?
PROCESS
Methods
Measures
How Done?
What Results?
ASQ - March 2007
© 2006 Whittington & Associates, LLC
OUTPUT
Deliver
what?
Slide 19
Process Effectiveness
•
•
•
•
•
•
•
•
View system as set of integrated processes
Understand their interfaces and interactions
Adopt the process approach for your audits
Add value by looking at more than conformity
Evaluate linked processes for “effectiveness”
Verify their controls and identify process risks
Determine any opportunities for improvement
Promote process view through audit methods
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 20
Summary
Difficult areas to audit:
1.
2.
3.
4.
5.
6.
7.
Undocumented Process
Legal Requirements
Resource Management
Continual Improvement
Preventive Action
Internal Audits
Process Effectiveness
Questions about auditing these or other areas?
ASQ - March 2007
© 2006 Whittington & Associates, LLC
Slide 21