Panel Testing, Full version
PANEL TESTING, Full Version (National, TOT and Higher Intermediate Level):
Purpose:
To provide a comprehensive overview of panel testing concept and
panel testing implementation principles
Module Time:
3 hours
Learning
objectives:
At the end of this module, participants will be able to:
 Give definition to panel testing
 Name advantages and disadvantages of panel testing
 Describe the scheme of a panel testing round
 Understand principles of scoring system; interpretation of
results and feedback
 Become familiar with registration forms used for panel
testing
Module Overview
Step
Time
Activity/
Method
Content
Resources
Needed
1
5 min
Presentation
What is panel testing?
Slides 3
2
5 min
Presentation
What is panel testing used for?
Slides 4 - 5
3
5 min
Presentation
Advantages and disadvantages
of panel testing
Slides 6 - 7
4
5 min
Presentation
Preparation of test smears
Slides 8 -9
Presentation
and
Demonstration
Validation of panel batches
Slides 10 - 18
5
40 min
6
5 min
Presentation
Panels’ composition
Slides 19 - 20
7
15 min
Presentation
Organization of a panel testing
round
Slides 21 - 27
8
10 min
Presentation
Analysis of results; scoring
system
Slides 28 - 31
9
10 min
Presentation
Interpretation and feedback
Slides 32 - 35
10
5 min
Demonstration
Forms
Panel Testing
Recording and
Reporting Forms
11
5 min
Summary
Key Messages
Slide 36-38
12
30 min
Exercise
Errors identification, scoring
and analysis of individual
Exercise notes;
exercise file.
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 1 of 10
Panel Testing, Full version
panel testing results
13
40 min
Exercise
Post-validation / analysis of
aggregate results
Exercise notes;
exercise file.
Material/Equipment Checklists

PowerPoint slides

Computer w/LCD projector

Flipchart

Calculators (scientific calculators or simple calculators that have a square root
function at least)

Handouts: one printed copy per participant

1.
2.
3.
4.

Panel Testing Recording and Reporting Forms (handouts for participants)
Validation Log for AFB Panel Testing Slide Batches at NRL
Panel Sets’ Logbook / Aggregate Results
Panel Testing Individual Results and Feedback Form
Panel Testing Report of Multiple Laboratories for District Supervisor & NTP
Exercise file and printed copies (one per participant of each of the 4 sheets of the file;
do NOT staple the sheets).
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 2 of 10
Panel Testing, Full version
Teaching Guide
Slide Number
1
Teaching Points
Module: Panel Testing, Full Version
DISPLAY this slide before you begin the module. Make sure
participants are aware of the transition into a new module.
2
Flipchart
Content Overview
(Suggested technique for presentation)
WRITE the content outline before beginning this session.
REFER to flipchart frequently to orient participants to where they
are in the module.
EXPLAIN that these are the topics that will be covered in this
module.
3
What is Panel Testing?
STATE that panel testing is an EQA method that is designed to
assess proficiency of an individual laboratory technician in
detecting, quantifying, interpreting and reporting AFB smear results
according to WHO/IUATLD guidelines, using a set of stained
and/or unstained sputum smears prepared by NRL.
4
What is Panel Testing Used For?
STATE the messages on the slide. Refer to the clarifying notes
below the slide.
STRESS that panel testing is the most effective tool to evaluate
proficiency and capacity, and most valuable with regard to training.
ADD that panel testing can also be useful additional to rechecking
for laboratories that rarely see positives as a “refresher” of the
appearance of a positive smear.
5
What is Panel Testing Used For? (2)
STATE the messages on the slide. Refer to the notes below the
slide.
Although panel testing is considered to be less effective than
rechecking (it does not monitor routine performance), it can be
more cost effective (i.e. in areas with low incidence of tuberculosis
or very few microscopy problems, or the only feasible option).
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 3 of 10
Panel Testing, Full version
Slide Number
6
Teaching Points
Advantages of Panel Testing
LIST panel testing strengths as stated on the slide. A rapid gross
overview of the main problems can be obtained at moderate cost.
7
Disadvantages of Panel Testing
STATE the message on the slide.
STRESS that panel testing requires additional resources; NRL staff
should be capable to handle the increased workload.
8
Preparation of Test Smears
CUSTOMIZATION: explain which type of panel slides are used in
your program.
STATE the message on the slide.
STRESS that panel sets are preferably specially manufactured.
Reusing stained patient smears is advisable only when resources are
extremely limited and NRL technical capacities and capabilities are
insufficient to prepare high quality standardized panel sets.
9
Manufacturing Smears for Panel Testing
CUSTOMISE this slide if the origin of panels is not your NRL.
STATE the message on the slide.
10
Validation of Panel Batches
STRESS that validation of the consistency of panel batches is a
must for any laboratory preparing panel smears.
EXPLAIN that validation of panel batches includes two parts, as
stated on the slide.
11
Pre-validation
DESCRIBE pre-validation steps as stated on the slide.
NOTE that if the consistency of a positive batch was considered
insufficient (the average minus 2 SD < 0), this batch can still be
used for another quantification target (less than 2 step difference),
provided that no negative smears appeared in the batch.
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 4 of 10
Panel Testing, Full version
Slide Number
12
Teaching Points
Normal Distribution
Country CUSTOMIZATION is recommended: Slides # 12 through
#15 are intended for NRL staff training only. Consider omitting
these slides if no NRL technical specialists are attended the
workshop.
NOTE that the next few slides will facilitate an understanding of
the standard deviation conception and its applicability to the
validation of AFB smear microscopy panel batches.
EXPLAIN that a normal distribution curve contains values that are
symmetrically distributed around the mean. The figure on the right
represents a normal distribution curve for a large number of
measurements of the same item. Values are plotted on the x-axis
with the frequency of the measurement plotted on the y-axis.
13 Flipchart
What is the Standard Deviation?
EXPLAIN the meaning of standard deviation and how it is
calculated.
USE flipchart to write down all steps as stated – better to do it in
advance prior to the training session in order to save time during the
presentation.
Step 1. Calculate the arithmetic mean.
Step 2. Subtract the mean from each observation. Square the
difference.
Step 3. Sum the squared differences.
Step 4. Divide the sum of the squared differences by n-1.
Step 5. Take the square root of the value obtained in Step 4. The
result is the standard deviation.
Two and three standard deviations can be computed simply by
multiplying the 1 SD value by 2 and 3 respectively.
14
Standard Deviation and Probability
STRESS that the standard deviation is a measure of the distribution
of values around the mean. The greater the spread of values, the
larger the standard deviation and the lower the precision. In the
laboratory we aim for as small a standard deviation as possible.
STATE the message on the slide.
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 5 of 10
Panel Testing, Full version
Slide Number
15
Teaching Points
Sample Form: Validation Log for AFB Panel Testing Slide
Batches (pre-validation)
Computer
SHOW the Pre-Validation Log form (slide).
STRESS that any laboratory preparing panels must keep the PreValidation Log electronically or in hard copy to record all batches
prepared and validated.
Flipchart
(OPTIONAL) REFER to Slide #13 notes above and USE flipchart
to DEMONSTRATE relevant calculations for panel batches as per
examples provided.
STRESS that intended positives should never be negative; intended
negatives should never be positive. A positive batch with deviating
average quantification can be used for another quantification target,
provided quantification differences do not reach 2 steps on scale.
DEMONSTRATE STDV&averageLog.xls file and, following
recommendations contained in the file, EXPLAIN the use of the
developed “Panel Test Log Validation” tool.
16
Post-validation
STATE the message on the slide.
STRESS that post-validation is needed mainly when high error
frequencies are observed for a particular slide batch.
17
Logbook of Panel Slides Sets / Post-validation
CUSTOMIZATION: you may want to replace this form by your
own format.
DEMONSTRATE the sample form of the panel sets’ logbook that
is used for post-validation purposes.
18
Coding of Panel Smears
EXPLAIN that the purpose of coding panel smears is twofold – to
prevent identification of a smear result by a tested laboratory
technician (thus to avoid reading bias) but to make identification of
a smear clear for the laboratory supervisor who is in charge of panel
testing.
MODIFY the example of a panel smear code as needed.
19
Panels’ Composition
STATE the message on the slide.
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 6 of 10
Panel Testing, Full version
Slide Number
20
Teaching Points
Examples of Panel Sets Compositions
GIVE examples of panel sets compositions as stated on the slide.
EXPLAIN why larger numbers of scanty and low positive smears
may be included. They represent more challenge, leading to a more
accurate, sensitive and discriminating test.
21
Getting Started: Issues to Consider
EXPLAIN that proper implementation of a panel testing program
requires a lot of efforts; there are many issues to be considered.
NAME those stated on the slide.
22
Implementation of Panel Testing
CUSTOMIZATION: Specify who is responsible for
implementation, in case it is not your NRL and Public Health
Directors.
STATE the message on the slide.
STRESS that a thorough planning is highly important for a
successful panel testing.
23
Sending Slides
STATE each program will need to determine the best mechanism
for delivering slides based on the services, regulations in force and
resources available. Reasonable turn around time is expected to be
between one week and one month, and must be set depending on
the local situation. Appropriate care should be taken to prevent
breakage when sending out panels.
NOTE that panel sets might be needed to return to NRL for review
after the panel testing exercise is completed at the peripheral level.
24
A Possible Scheme of a Panel Testing Round
CUSTOMIZE if needed by indicating who is responsible at which
level of your country, if not the NRL and intermediate laboratories.
DESCRIBE a possible scheme of a panel testing round as shown in
the slide. White color arrows show panel sets flow from NRL to
peripheral laboratory technicians via intermediate laboratories. The
broken type arrows of blue color demonstrate the panel testing
results’ reporting flow.
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 7 of 10
Panel Testing, Full version
Slide Number
25
Teaching Points
Performing a Panel Test Round
CUSTOMISE the first bullet to match the country policy on the
frequency of panel testing rounds in the country.
STATE the message on the slide.
STRESS that each individual technician must examine smears and
report results separately.
26
Individual Results of Panel Testing / Feedback
Form
CUSTOMIZATION: you may want to replace this form by your
own format.
DEMONSTRATE the sample laboratory form designed to record
individual results of panel testing and feedback.
EXPLAIN that the form and/or a separate accompanying letter
should provide the participating laboratory with the detailed
instructions regarding the staining, time allowed for technicians to
complete panel testing exercise, etc. The feedback section should
present carefully considered conclusions and recommendations, not
simply a bunch of scores.
27
Management of Panel Testing During a Supervisory Visit
EXPLAIN the message on the slide.
28
Analysis of Panel Testing Results
STATE the message on the slide.
CUSTOMIZATION: mention the value of a successful score as
determined by NTP/NRL
29
Types and Classification of Errors
EXPLAIN the types of errors as shown in the table.
30
Example of Panel Testing Scoring
GIVE an example of a panel testing scoring system as stated on the
slide.
31
Analysis of Panel Testing Results
STATE the message on the slide.
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 8 of 10
Panel Testing, Full version
Slide Number
32
Teaching Points
Panel Testing – Interpretation of Results
STATE the message on the slide.
33
Feedback to Laboratories on Panel Testing Results
STATE that feedback is always required and should be timely and
confidential. Participants should be informed of their own result
and how it compares to the aggregate performance of other
laboratories. Reports should provide appropriate background
information and recommendations and not simply scores. A visit to
the centers scoring poorest, with on-site identification of the
problems and remedial action, is required.
34
Panel Testing Aggregate Results of Multiple Laboratories
CUSTOMIZATION: you may want to replace this form by your
own format.
DEMONSTRATE the sample reporting form on panel testing
aggregate results of multiple laboratories.
35
Panel Testing Aggregate Results Report: Example
DISCUSS the Panel Testing Report presented (see slide notes).
Panel Testing
Recording and
Reporting
Forms
(electronic
files and
paper-based
handouts)
Panel Testing Forms
36
Conclusion:
(OPTIONAL) SUMMARIZE panel testing related recording and
reporting forms.
STRESS that proper data collection and analysis is a critical
component of a successful EQA program and laboratory quality
improvement.
STATE the message on the slide.
37
Key Messages (I):
STATE the message on the slide.
38
Key Messages (II)
STATE the message on the slide.
ANSWER any questions the participants may have.
Exercises
MOVE to the exercise section below.
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 9 of 10
Panel Testing, Full version
NOTES TO THE EXERCISE “ERRORS IDENTIFICATION, SCORING AND
ANALYSIS OF INDIVIDUAL PANEL TESTING RESULTS”
The purpose of this exercise is to develop participants’ analytical skills in examining
individual panel testing results and making recommendations for appropriate actions to
improve performance. Participants should demonstrate an understanding of smear
microscopy errors classification principles as well as results scoring rules. Participants
will be asked to review the completed form that contains the panel test results of the
Technician from the Laboratory A, compare the Technician’s results with those of NRL,
identify errors, if any, record each error in an appropriate abbreviated form, score results,
decide whether the Technician has successfully passed the panel testing exercise or failed
By doing this exercise participants will be taught how to correctly use the proposed
recording form. It is also important that participants will practice their skills in providing
written feedback and recommendations.
Participants are given 20 minutes to read the task and complete the exercise individually,
in written form. Next, the Trainer and participants discuss the exercise in a group after
everyone has completed the task (10 minutes): the Trainer asks a participant to provide a
verbal answer to the questions stated in the exercise and encourage other participants to
add their comments and share their opinion with the group. The Trainer may also want to
collect all completed forms to check participants’ answers and return the forms with
remarks afterwards.
NOTES TO THE EXERCISE “POST-VALIDATION / ANALYSIS OF
AGGREGATE RESULTS”
The purpose of this exercise is to develop participants’ analytical skills in examining
aggregate panel testing results and post-validating panel batches. Participants are
provided with the completed form that represents aggregate results from 10 Laboratory
Technicians. Participants are asked to compare the Technicians’ results against NRL
results and make conclusions about the consistency of slide batches prepared at NRL.
Participants should also make a judgement on how many Technicians have successfully
passed the panel test. Participants should first also select peripheral laboratories to be
visited to investigate errors. This exercise is also good for gaining a better understanding
of the panel smears coding principles and the importance of proper registration of panel
sets.
This exercise is more complicated than the previous one and therefore will require more
time (40 minutes in total); it should go after the first exercise on the analysis of individual
panel testing results. The Trainer should instruct participants to read the task carefully in
order to be able to provide full answers as required. Participants are given 25 minutes to
complete the exercise in written and then 15 minutes are devoted to the discussion of the
exercise in the group.
REMARK: The exercises are attached as a separate file and contain tasks and answer
sheets. Please refer to the exercise body for detailed descriptions and instructions.
Participants should first be provided with tasks sheets only. Answer sheets can be
distributed as reference materials after everyone completes the exercise and answers are
discussed in the group.
TRAINER’S GUIDE
AFB Smear Microscopy EQA WIB
Page 10 of 10