Topics
 Assessor seminar feedback
 Laboratory feedback
 Sampling of tests for assessment
 New/revised requirements of 17025
 Statistics on non-conformities
 Empirical in-house methods
1
Feedback last assessor seminar
 Fulfilling assessors’ need and expectation
 Duration
 Gain from the seminar
2
Feedback last assessor seminar
 127(110) assessors attended
 84(48) evaluation forms returned
 Fulfilling assessors’ need and expectation
 0 (0) % not really, 23 (31)% adequately, 57
(65) % well, 20 (4) % very well
 Duration
 0 (0) % too long, 96 (98) % suitable, 4 (2) %
too short
3
Feedback - gain
 112 comments received
 Grading of non-conformities (9)
 Views and experience of assessors
(17)
 Latest development of HKAS (24)
 Technical issues, eg PT, MU, etc (6)
4
Feedback –
suggestions on future activities
 24 suggestions received
 More assessor workshops
 Longer discussion
 More technical seminars
5
Laboratory Feedback
 September 2005
 All accredited organisations
 Accredited labs – 128
 Return (labs) – 55 = 43%
6
Laboratory Feedback
- Degree of rigor
 Substantially relaxed – 0%
 Slightly relaxed – 29.5%
 Remained unchanged – 67.2%
 Slightly tightened – 3.3%
 Substantially tightened – 0%
7
Laboratory Feedback
- Degree of rigor
 Depending on the availability of
technology available in the market, some
equipment, specifications or requirements
stated in testing standard that cannot be
fulfilled should be duly accepted. Surely it
can be reviewed again per the
advancement of technology.
8
Laboratory Feedback
- Degree of rigor
 More flexibility to be given for quoting
accreditation to a standard with
modification for those modified method
verified with same level of performance
characteristics.
9
Response
Accreditation body is not in a position to
change requirements and/or specifications
given in standards.
Review the requirements and/or
specifications, based on the intent, to
determine if a more flexibility interpretation
could be justified.
If an apparent error is found in the standard,
the standard writing body should be
consulted.
10
Laboratory Feedback
- Degree of rigor
 Consistency on technical/quality
requirements between different assessors
should be aligned.
 Sometimes, the corrective actions taken
for non-conformities raised in the
previous assessment may not be accepted
in the next assessment.
11
Response
HOKLAS officers will discuss with technical
assessors during assessments to ensure that
requirements are consistent
Annual assessors conclave to discuss
technical matters to ensure consistency
12
Response
Perception of inconsistency may be due to
misinterpretation/misunderstanding of the
assessors’ requirements.
More detailed explanation during assessments
and clarifications by HOKLAS officers
during or after assessments to be given
Unavoidable due to rapid development in
some areas
13
Laboratory Feedback
- Degree of rigor
 It can help to maintain lab standard.
 I like officer be tighten on the quality
system.
14
Laboratory Feedback
- Degree of rigor
 It seems that the assessment is too
concentrated on minor points rather than
major issues.
15
Response
Focus on processes and competencies rather
than quality manual
16
ISO/IEC 17025:2005
• Published on 15 May 2005
• 8th edition of HOKLAS 003
published in October 2005
• Full implementation by 15 May 2007
by accredited laboratories
• No extra assessment to be conducted
• Amendment mainly on Section 4
(Management requirements)
17
HOKLAS 003 8th edition
• Section 1 Introduction - Updated
• Section 2 Normative references –
New
• Section 3 Terms and definitions –
New
• Sections 4 & 5 – revised as per
ISO/IEC 17025:2005
• Section 5 Technical requirements –
only two revised/new clauses
18
Section 5 HOKLAS 003 8th edition
Clause 5.2.2 Personnel
• …. The effectiveness of the training
actions taken shall be evaluated.
Clause 5.9.2 Assuring the quality of test and
calibration results
• Quality control data shall be analysed and,
where they are found to be outside predefined criteria, planned action shall be
taken to correct the problem and to prevent
incorrect results from being reported..
19
HOKLAS 003 8th edition
• Section 1 Introduction
Compliance with regulatory and safety
requirements on the operation of
laboratories is not covered by this
International Standard.
• Section 5.3.H
It should be emphasized that it is the
responsibility of the laboratory to ensure
that its operation is in compliance with all
applicable regulations, particularly those
related to laboratory safety.
20
HOKLAS 003 8th edition
•
Section 4.9.H Control of nonconforming testing
and/or calibration work
Examples –
1. environmental conditions exceeded limits;
2. duration of conditioning of test specimens not
complying with specifications;
3. max. holding times of samples exceeded;
4. tests performed using instruments with expired
calibration;
5. QC acceptance limits exceeded;
6. performance in PT unsatisfactory
21
HOKLAS 003 8th edition
• Section 4.13.H Control of records
The retention period of at least 3 yrs for
equipment records and lab. operation
procedures shall be counted from the date
on which the use of the equipment or the
operation procedures have been
discontinued respectively. Similarly, the
retention period of at least 3 yrs for
personnel records shall be counted from
the date of departure of the staff member
concerned.
22
Sampling of tests for assessment
ISO/IEC 17011:2004
Conformity assessment –
General requirements for accreditation
bodies accrediting conformity assessment
bodies
23
Sampling of tests for assessment
Cl. 7.5.6 Preparation for assessment
The accreditation body shall establish
procedures for sampling (if applicable)
where the scope of the CAB covers a
variety of specific conformity assessment
services. The procedures shall ensure that
the assessment team witness a
representative number of examples to
ensure proper evaluation of the
competence of the CAB
24
Sampling of tests for assessment
• Cl. 7.6.1 Document and record review
The assessment team shall review all relevant
documents and records supplied by the CAB… to
evaluate its system, as documented, for conformity
with the relevant standard(s) and other
requirements for accreditation
• Cl. 7.7.3 On-site assessment
The assessment team shall witness the performance
of a representative number of staff of the CAB to
provide assurance of the competence of the CAB
across the scope of accreditation.
25
Sampling of tests for assessment
• HKAS Management Manual App 9.4A5
…Emphasis should be given to new
activities to be accredited and activities
and aspects which are known to be
problematic, e.g. activities with poor
proficiency testing results or performed in
accordance with standards which are
known to have different interpretations,
subjects of complaints, aspects where
many non-conformities were identified in
previous assessments, etc….
26
Sampling of tests for assessment
• HKAS 008 Guide for HKAS Assessors and Lead
Assessors Clause 7.3.1 Observing of the
Organisation’s Practice
The assessment team should witness some or all of
the tests … within the scope of accreditation.…
Even if a particular test … is too lengthy to be
performed in its entirety during the visit, it should
be possible for at least the most critical phases of
the test … to be performed…Depending upon the
circumstances, samples or specimens used in such
tests … may be conveniently drawn from the
normal work throughput … or provided by a
member of the assessment team through
arrangement with the team leader.
27
Sampling of tests for assessment
• Difficult to have hard and fast rules
• Random sampling is not the most
appropriate
• Priority for those not assessed before
• At least select a representative test in each
test area
28
Sampling of tests for assessment
• Like driving test – select those that need
special skills and knowledge
• Like an examination, select those tests
which are most indicative of lab’s
competence
• Like cooking class, if the test is solely for
demonstration purpose, skip those noncritical steps
• More than one test can be performed at a
time
• Prior arrangement with the lab is helpful
29
Laboratory internal validation and
verification
Existing information
Laboratory
requirement
Verification
Fully validated standard methods
(have been studied in a collaborative (Secondary
validation)
trail)
Standard methods – amplifications
and modifications e.g. new
instrument
Standard methods – outside their
intended scope
Laboratory – developed and nonstandard methods
Validation
Validation
Validation
(Primary validation)
30
Definition
• Validation is the confirmation, through provision of
objective evidence, that the particular requirements
for a specific intended use or application have been
fulfilled (ISO/IEC 17025 clause 5.4.5.1 and ISO
9000:2000)
• Verification is the confirmation, through provision of
objective evidence, that specified requirements have
been fulfilled. (ISO 9000:2000)
• Objective evidence is data supporting the existence
or verity of something (ISO9000:2000)
31
Standard methods shall be
verified for
1.
Procedures conformity with the standard method
2.
Availability and suitability of equipment and reference
materials/standard, reagents, etc
3.
Appropriateness of environmental conditions
4.
Competence of testing staff
5.
Capability of lab of achieving the defined method
performance characteristics
32
Main validation parameters
1. Applicability
2. Selectivity
3. Calibration and linearity
4. Trueness
5. Precision
33
Main validation parameters
6. Recovery
7. Working concentration and
8.
9.
measurement range
Limit of Detection (LOD) and
Quantitation (LoQ)
Matrix variation
10. Measurement Uncertainty
34
Common insufficiencies
1. Method bias not established or demonstrated
2. Precision data derived from short term results
and do not cover major variations
3. Detection limits wrongly determined
4. Major
components
omitted
in
measurement uncertainty estimation
35
Empirical Methods
 Definition – the quantity estimated is simply the result
found on following the stated procedure.Hence, method
bias is zero and matrix variation is irrelevant. However,
laboratory bias need to be assessed. Apart from these,
method validation similar to other rational methods.
Empirical methods are often defined by standard
methods or legislation.
36
In-house Methods
 Definition –methods other than standard
methods and which are designed/developed by
the laboratory.
 Full method validation required
37
In-house Empirical Methods
 For some measurands (analytes), their values are defined
by the procedure used, which may be given in standard
methods, in legislation or by users of testing service.
Many commonly encountered measurands determined
by empirical methods have established meanings, e.g.
BOD, COD, SS, VS, etc.
38
In-house Empirical Methods
 For these measurands, in-house empirical methods need to
show comparability with the those defined in well recognised
standard methods.
 Very often, in-house empirical methods are modified standard
methods. Ensure the modifications do not affect the values
obtained, based on professional judgement and validation
data (e.g. CRM, PT, comparison with relevant standard
methods, etc.).
39
In-house Empirical Methods
 May need to define the measurands or give key
experimental conditions in test reports, e.g. extraction
conditions for metals (temperature, duration, acid
strength, etc.), drying temperature of suspended solids,
etc.
 Customer specified in-house empirical methods –
precision need to be demonstrated
40