Overcoming the challenges associated with Proficiency Testing (PT)
schemes for online chlorine analysis
Speaker / Author: P.J. Lems
Rand Water
PO Box 3526, Vereeniging, 1930, South Africa
e-mail: [email protected]
Phone: 083 566 2176 Fax: 016 455 2055
Abstract
Proficiency Testing (PT) and Inter Laboratory Comparison (ILC) schemes are Quality
Assurance (QA) tools used by laboratories to demonstrate that they are competent to execute
analytical methods. Establishment of Rand Water’s online testing capacity required the
development of suitable PT/ILC schemes to ensure the accuracy of the data. Over the past
four years Rand Water has implemented and managed a PT scheme for online pH and
conductivity analyses. The scheme has now been extended to online chlorine analyses.
Establishment of the PT/ILC schemes for online analyses presented a number of challenges.
These included: 1) difficulties associated with the introduction of external control samples
into online systems; 2) the absence of on-site analysts to receive and process control samples;
3) the distribution of analysers over an extensive reticulation system and 4) the need to
measure volatile parameters such as disinfectant residuals.
These challenges were addressed by using portable instruments, rather than reference samples
to compare the performance of online analysers. The principle was that a highly trained
analyst would travel from site to site and compare measurements made with a single portable
instrument against data recorded by online analysers. This presentation will consider the
challenges encountered in the establishment of this system and present examples of typical
data sets generated by the scheme. It will also attempt to highlight and discuss deviations
from more conventional PT schemes.
1. Introduction
In the last five years Rand Water has invested a great deal of resources in upgrading and
improving the online analytical analysis of its water quality. Twenty new online Laboratories
had been constructed and the project should reach completion by end 2013. Upon completion
Rand Water will have one of the most advanced online water quality monitoring processes in
Africa. A total of 350 Quality Assurance analysers will analyse key parameters such as pH,
conductivity, turbidity, ammonia and the chlorine species which include free and total
chlorine and monochloramine on 71 pipelines delivering 3800Ml of water every day.
With the investment and improvement in Rand Water’s analytical online measurements
capabilities there is a need for better quality assurance of analysis produced. Thus the
decision was made that the new online facilities need to be accredited under ISO 17025:2005.
One of the requirements for Accreditation is the participation in a PT/ILC.1
Test and Measurement Conference
For conventional laboratories PT/ILC are QA tools that are used to show competency in the
analysis of analytical methods. ISO 17043:2010 specifies general requirements for the
competence of providers of proficiency testing schemes and for the development and
operation of proficiency testing schemes2. Implementation of the standard towards the online
measurement applications has proved to be problematic and even ILAC P9:11/2010 clause
4.6 recognises that not all areas are equally suited for PT/ILC and thus recommends that the
laboratories discuss and agree on suitable alternative means by which the performance can be
assessed and monitored.3
For this reason Rand Water is developing in-house PT/ILC schemes that are modified to fulfil
the needs of the online analysers. Rand Water has successfully implemented a PT/ILC for
online analysis of pH and Conductivity with only minor deviations from the requirements
With the need to expand the PT/ILC to include the chlorine analysis the preliminary
assessments highlighted numerous challenges and it was found that the conventional
preparation and distribution of control samples would not yield a workable solution. An
alternative method of determining the assigned values used for comparison had to be
determined.
The most plausible solution is to use a portable instrument as a reference. A single analyst
would make use of a validated instrument to perform analysis on water samples as it is
simultaneously analysed by the online analysers. Paired analysis would be used to determine
the deviation from the assigned values as calculated with the portable analyser.
2. Identification of challenges that prevents PT/ILC for chlorine analysis
The challenges that were highlighted by the initial assessment are primarily linked to the use
of an external sample as a reference. The first limitation that was identified deals with the
design of the online analysers and its inability to analyse external samples. Next two items
identified was the volatility of chorine and coupled with that the distribution of samples over
an extensive reticulation network. Finally the absence of an onsite analyst to receive and
process the control samples was identified.
2.1
Challenge of the online analyser design
The online analysers used are not designed for manual or batch operations and as such do
necessarily not provide for the introduction of an external sample. Within Rand Water three
different types of analytical methods are used for online chlorine determination and each has
its unique challenges.
2.1.1 Amperometric analyser
The amperometric analysers manufactured by Capital Controls, the 1870E performs
continuous measurements whereby an acetic buffer is continuously added to the sample as it
enters the analyser. It was considered that the installation be modified to allow for an external
sample to be fed into the analyser. However taking into consideration the volume of samples
required for this type of analysis, the number of analysers and the modifications that had to
be made to introduce these samples it was found to be unfeasible.
Test and Measurement Conference
2.1.2 Colorimetric analyser
The next type of analyser is a Hach CL17 colorimetric analyser. These are new analysers that
have been installed in the newly upgraded online laboratories. The analyser performs an
analysis every two minutes. A sample flows into the analyser where buffer and DPD
indicator reagents are added.4 Colour reaction is allowed to take place and a measurement is
made. This analyser could be capable of measuring an external sample, however during the
commissioning of the analysers in the online facilitates it was found that the variations in the
flow rate had a negative impact on the analysers performance. To rectify the problem the
suppliers recommended that a constant head device be fitted to the installation to minimise
the flow variations. This addition however removed the possibility of having external
samples analysed by the analysers.
2.1.3 Polarographic analyser
The final type of chlorine analyser used in Rand Water is primarily found in the distribution
network. It is an ATI Q45 analysers that utilizes a membrane-covered polarographic sensor
that does not require the addition of chemical reagents. As this analyser has a probe sensor
that could be removed it is the only analyser that could allow for a manual sample to be
analysed.
2.2
Volatility of chlorine
The volatile nature of the chlorine means that the concentration would decrease until there
were no residual left. This makes it impractical to prepare a control sample in advanced and
send it to the participant. The volatility also affects the availability of Certified Reference
Materials (CRM). The CRM’s that was identified includes a sample matrix and concentrated
chlorine sample that has to be prepared prior to analysis in accordance with the suppliers
specifications. This is again not a process that is accurately performed in the field
environment.
2.3
An extensive reticulation network
The third challenge links to the volatile nature of the chlorine samples and that a control
samples would have to be transport over an extensive reticulation system. Some of the key
analysers are installed on the ends of the distribution which covers 3000 km of pipelines.
With such great distances between sites and the time taken to travel it does indicate that a
PT/ILC would take a couple of days to be completed, as is the case with the pH and
conductivity PT/ILC. With chlorines low stability in natural waters it would be impractical to
store control samples for extended periods.
2.4
Absence of an on-site analyst
The last challenge identified was the absence of an on-site analyst to receive and process the
control samples. The online Analysers are installed and operated in such a manner that very
little human interaction is required. Dedicated maintenance personnel are responsible only for
the maintenance of the analyser. As per the different requirements the maintenance teams will
clean any deposits from the analysers and replenish the chemicals required. Since these
maintenance staff do not have any experience with the actual preparation and analysis of
Test and Measurement Conference
samples it was thought that to have them analyse samples would introduce an error with is
normally not associated with the online analysis.
3. Development of the PT/ILC method for online chlorine analysers
With the development of the PT/ILC for online chlorine analysis two key areas had to be
addressed. The first deals with the validation of the analysers that would be used as the
reference and the second are the actual process followed to conduct the PT/ILC. The
processes followed for the two activities will be discussed and the results are included.
3.1 Instrument validation.
The key area that was identified with regards to a good PT/ILC scheme was the accurate
determination of the assigned value. This approach makes use of an instrument that was
validated and is check using NIST traceable standards, to determine the assigned value.
Standard methods list eight different methods for the determination of chlorine. Analysis of
each method revealed that the DPD colorimetric method appears the most feasible for field
analysis. The main reason for selecting this method is the simplicity thereof. The more
sensitive methods such as the amperometric titrations cannot be performed with the required
accuracy in a field environment. Thus the DPD colorimetric method was selected for the
determination of free and total chlorine residuals. The instrument selected for this method is a
Hach DR890 colorimeter.
3.1.1 Principal of method.
N,N-diethyl-p-phenyldiamine (DPD) is the reagent that is added to the sample. Chlorine
oxidizes this DPD to form two possible oxidation products, a magenta coloured compound
known as Wurster Dye and a colorless imine compound:
Figure 3: DPD-chlorine reaction products
The intensity of the colour is directly proportional to the amount of chlorine present in the
sample. The sample are analysed photometrically at wavelengths ranging from 490 to
555nm. 5 The supplier’s method for the DR890 does provide separate DPD SwifTest reagents
that are used for free and total chlorine determination.
Test and Measurement Conference
3.1.2 Validation process
The Hach DR890 colorimeter had been validated in accordance with the work instruction
used by the Rand Water inorganic Laboratory.6 The validation was conducted on both
chlorinated and chloraminated water sources. The validation was conducted over three nonconsecutive days and each day the sample analysis was repeated seven times.
The validation included the following analysis:
1. Hach Free Chlorine Dispenser- reagents weighed
2. Hach Total Chlorine Dispenser- reagents weighed
3. DR/Check Absorbance standard, Lot A3085, Exp. Mar 2015 - absorbance
standard analysis at different wavelengths (420 λ, 520 λ, 560 λ, 610 λ)
4. DPD chlorine LR standards kit, lot A3086A, exp. March 2015- analysis of blank,
std1,std2 and std3
5. Blank Millipore water with Free DPD reagent
6. Blank Millipore water with Total DPD reagent
7. Blank Millipore water with chlorine spike
8. Free chlorine water sample from Vereeniging A19 pipeline water 9. Free chlorine A19 sample with spike
10. Free chlorine A19 sample diluted 10 times
11. Low level free chlorine sample from Palmiet A6 pipeline (pre-chloramination)
12. Total chlorine Palmiet O2 pipeline water(chloraminated)13. Total chlorine Palmiet O2 sample diluted 10 times
3.1.2 Validation results
3.1.2.1 Detection limits (LOD) and limits of quantification (LOQ)
The LOD and LOQ were calculated by combining the three days results for the blank
analysis performed on reagent grade water with the addition of free and total reagents.
Table 1. Summary of the calculated average LOD and LOQ
Blank Millipore water with free
DPD
Mean
0.011
Standard deviation 0.005
LOD
0.026
LOQ
0.059
Blank Millipore water with total
DPD
Mean
0.014
Standard deviation 0.005
LOD
0.029
LOQ
0.064
From table 1 it can be seen that the detection limits are well below the minimum residual
concentration of 0.2 mg/l free chlorine at are required at the point of delivery.7
3.1.2.2 Precision
The precision was determined by utilising the % RSD of the various samples analysed. The
table below contains the average value for each sample analysis performed over the three
days as well as the spike precision
Test and Measurement Conference
Table 2: Determination of the precision
Free chlorine VG site water A19
Day 1
Day 2
Day3
Mean
0.95
1.30
1.12
Sigma
0.01
0.02
0.01
% RSD
1.22
1.29
0.85
Low level free chlorine Palmiet A6
Day 1
Day 2
Day3
Mean
0.65
0.85
0.89
Sigma
0.01
0.01
0.01
% RSD
1.06
1.33
0.65
Free chlorine VG sample with spike
Day 1
Day 2
Day3
Mean
1.69
1.95
1.80
Sigma
0.02
0.03
0.01
%RSD
1.41
1.34
0.68
Total chlorine Palmiet suction kiosk O2
Day 1
Day 2
Day3
Mean
1.74
1.78
1.73
Sigma
0.03
0.02
0.02
% RSD
1.47
1.10
1.15
The %RSD values as shown in table 2 are all less that 2% with a mean of 1.12%
3.1.2.3 Accuracy
The method adopted in the validation for the determination of accuracy was the use of NIST
traceable standards, Absorbance standards and % spiked recoveries. The NIST traceable
standard is a chlorine standard from Lot Nr. A3086A and expires March 2015. The
absorbance standard, CAT No. 27639-00, is a set of three standards that are analysed at four
different wavelengths. The spiked samples were Vereeniging A19 water sample that was
spiked with a chlorine stock solution.
Table 3. Calculated % Accuracy for each sample
STD 1
Accuracy %
ABS standard at 420 λ wavelength
STD 2
STD 3
98.86
Accuracy %
100.64
Accuracy %
100.92
STD 1
Accuracy %
ABS standard at 520 λ wavelength
STD 2
STD 3
99.17
Accuracy %
100.62
Accuracy %
100.91
STD 1
Accuracy %
ABS standard at 560 λ wavelength
STD 2
STD 3
97.95
Accuracy %
99.73
Accuracy %
100.26
STD 1
Accuracy %
ABS standard at 610 wavelength
STD 2
STD 3
98.67
Accuracy %
99.63
Accuracy %
100.11
STD 1
Accuracy %
Chlorine standard (Lot Nr. A3086A)
STD 2
STD 3
102.72
Accuracy %
102.30
Accuracy %
102.69
Test and Measurement Conference
Day 1
Spike recovery %
Free chlorine VG sample with spike
Day 2
Day3
94.06 Spike recovery % 94.24 Spike recovery %
94.87
The average accuracy that was determined using the standards was 100.35% and an average
spiked recovery of 94.39%. From the analysis of the validation results the method had been
found to be fit for purpose.
3.2
PT/ILC method
Having established that the portable method for determination of free and total chlorine is fit
for purpose the method for comparative analysis had to be developed
3.2.1 Apparatus used during the PT/ILC run
1.
2.
3.
4.
5.
6.
7.
Hach DR 890 colorimeter serial nr: 120290C88637
Hach Free Chlorine SwifTest dispenser
Hach Total Chlorine SwifTest dispenser
Glass beakers (100 mL)
10ml Eppendorf research plus pipette
Eppendorf Tips (10ml)
Hach Sample cell (25ml)
It should be noted that all glassware used are free of chlorine demand and disinfectant
chlorine species.
3.2.2
Verifications to be performed in the laboratory prior to testing
Prior to the start of the PT/ILC a set of NIST traceable standards that are kept in the
controlled environment of the laboratory are analysed on the colorimeter. The standard
confirms that the instrument response is within the tolerance range as listed on the certificate
of analysis and that the performance is still the same as during the initial performance
evaluation.
Test and Measurement Conference
Concentration (mg/l)
Validation of Chlorine Standard
Lot nr A3086A ,Exp. March 2015
1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
STD 0.21 (±0.09)
STD 0.87 (±0.10)
STD 1.52 (±0.14)
Dates analysis performed
Figure 1: Validation of Primary chlorine Standard
Expired NIST traceable standard are also verified against the current principal NIST
standards. These expired standards are transported with the analyser and are used to ensure
that the instrument is not damaged during transportation. These travel standards are analysed
before and after the comparative analysis for a specific site is conducted.
Validation of Chlorine Standard
Lot nr A8315 ,Expired
1.8
Concentration (mg/l)
1.6
1.4
1.2
1
0.8
STD 1 (0.20mg/l)
0.6
STD 2 (0.86mg/l)
0.4
STD 3 (1.53mg/l)
0.2
0
Analysis per date
Figure 2: Validation of travel chlorine Standard
Test and Measurement Conference
3.2.3 Performing the measurement
At the participant site the travel standards are analysed prior to any analysis being conducted.
The data is collected in a control chart that is used for monitoring of trends.
Concentration (mg/l)
Control chart for standard verification
1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
UCL
LCL
UCL
LCL
UCL
LCL
1
3
5
7
9
STD 1 (0.20mg/l)
STD 2 (0.86mg/l)
STD 3 (1.53mg/l)
11 13 15 17 19 21 23 25 27 29 31
Number of analysis
Figure 3: Control chart of chlorine standards
The glass beaker and sample cells are rinsed with sample to be analysed. A new sample is
collected in the 100ml glass beaker. The Eppendorf pipette is used to pipette the required 10
ml of samples into the sample cell. The blank sample is capped, the sample cell wiped clean
and placed in the analysers. The instrument is then zeroed to compensate for interferences
from oxidized forms of manganese in the water.8 Once the instrument zero has been
established a new sample is to be collected.
The sampling has to be timed such that it is representative of the sample analysed by the
online analyser. For both the Capital Control and ATI the speed of response has to be taken
into consideration when sampling. The instrument reading that corresponds to the particular
sample will only be available after approximately 1½ to 2 minutes.9 This is due to the
continuous flow of sample and the time to displace the previous sample. For the Hach
analysers the instrument’s waste line is visually observed for the presents of water. Once
water is wasted it indicates that the analyser is taking a sample. The analysis is performed in
triplicate to compensate for the deviations that may occur during the sampling.
Once a sample is collected it is pipetted into the sample cell. The DPD free or total Chlorine
SwifTest is inverted, the reagents are added, and the sample cell is capped and wiped clean.
With Free chlorine the sample is analysed immediately while for total chlorine it is
recommended that the analysis is performed 2 minutes after the addition of the reagent.8 The
measured values for both the analysers are recorded on the result sheet.
Prior to departure that travel standards are again measured to ensure the instruments
performance during the PT/ILC are within the expected results.
Test and Measurement Conference
4.1 Results from PT/ILC
The three results from each analyser are used in the paired analysis. Figure 4 represents the
paired analysis data of each of the analysis. The difference between the online analyser and
portable instruments are calculated and plotted. The value from the portable instrument is
used as the assigned value for each of the comparisons.
Difference from assigned value (mg/l)
Paired analysis of results
0.40
0.20
0.00
-0.20
Comparison 1
-0.40
Comparison 2
Comparison 3
-0.60
-0.80
Participant confidential identification
Figure 4: Paired analysis of results
The f and t- test was used to determine whether the means form the two analysed samples
would be statistically comparable. The data from the F-Test in Table 4 indicates a variance in
the difference. This is to be expected as it is different methods that are used.
Table 4. F-Test results
F-Test Two-Sample for Variances
Mean
Variance
Variable 1
1.56
0.13
Variable 2
1.60
0.048
Observations
df
F
P(F<=f) one-tail
F Critical one-tail
27
26
2.83
0.0051
1.93
27
26
The calculation of the T-test value in table 5 shows that with t < t crit there is no statistically
significant difference in the means of the two sets of results.
Test and Measurement Conference
Table 5. t-Test results
t-Test: Paired Two Sample for Means
Mean
Variance
Observations
Pearson Correlation
Hypothesized Mean Difference
df
t Stat
P(T<=t) one-tail
t Critical one-tail
P(T<=t) two-tail
t Critical two-tail
Variable 1
1.56
0.13
27
0.80
0
26
-0.76
0.22
1.70
0.45
2.05
Variable 2
1.60
0.048
27
Since the analysis is all performed on different samples it is difficult to compare readings. To
overcome this the mean deviation from the assigned value for each participant is calculated.
Using the calculated mean deviation values a form of Z-score could be calculated as follows:
Z
x  t
With: x = Individual mean deviation
t = Overall Mean of the deviations
o
 o = Standard Deviation
Z-score
2.00
1.50
1.00
Z - Score
0.50
0.00
-0.50
-1.00
-1.50
-2.00
-2.50
CL1
CL2
CL3
CL4
CL5
CL6
CL7
CL8
CL9
CL10
CL11
CL12
CL13
CL14
CL15
CL16
CL17
CL18
CL19
CL20
CL21
CL22
CL23
CL24
CL25
CL26
CL27
CL28
CL29
CL30
-3.00
participant confidential identification
Figure 5: Z- Score analysis of results
Test and Measurement Conference
Using the following criteria the analysers can be classified and appropriate actions can be
taken in areas were results is questionable.
Z- Score = 0 ↔ 1
Excellent
Z- Score = 1 ↔ 2
Acceptable
Z- Score = 2 ↔ 3
Questionable
Z- Score = 3+
Unacceptable
5. Conclusion
With a system where assurance on the validity of the chlorine analysis could not be given this
PT/ILC is found to be an effective QA tool. The method used for the chlorine analysis will
be accredited in the near future to increase the validity and confidence of the assigned values
that are determined.
The analysis of the data does confirm the areas that are generally considered to be more
problematic are in fact giving higher deviations.
The research and implementation that were conducted during this method development also
highlighted areas that were previously unknown. The main area is the differences in the total
chlorine SwifTest method as used in this PT/ILC and the KI addition method that are used at
certain sites. Performing both the methods on the portable instrument yields significantly
different values.
6. References
1.
ISO 17025:2005 General requirements for the competence of testing and calibration
laboratories
2.
ISO 17043:2010 Conformity assessment – General requirements for proficiency testing
3.
ILAC –P9:11/2010 ILAC policy for participation in proficiency testing activities.
4.
CL17 Chlorine Analysers user manual, November 2008, Edition 5.
5.
Danial L. Harp, Current technology of chlorine analysis for water and waste water
Technical information series – booklet 17, 2002, Hach company.
6.
Chemistry Method validation guidelines, Work instruction no.2.1.1.03.1, 24 July2012.
7.
World Health Organisation, Guidelines for drinking water quality, 4th edition. 2011.
8.
Standard methods for the examination of water and waste water, 20th edition, 1998,
Published by the American public health association, American water works
association and Water environment federation, edited by Andrew D. Eaton et al.
9.
http://www.cftechflowmeters.co.za/documents/capitalControls/210-0001.pdf
Test and Measurement Conference