Journal of Medical Microbiology (2015), 64, 605– 609
DOI 10.1099/jmm.0.000064
Screening urine samples for the absence of
urinary tract infection using the sediMAX
automated microscopy analyser
Rosanne E. Sterry-Blunt,1 Karen S. Randall,1 Michael J. Doughton,1
Sani H. Aliyu2 and David A. Enoch2
Correspondence
Rosanne Elizabeth Sterry-Blunt
1
Clinical Microbiology & Public Health Laboratory, Peterborough and Stamford Hospitals NHS
Foundation Trust, Peterborough City Hospital, Bretton Gate, Peterborough PE3 9GZ, UK
[email protected].
2
PHE – Public Health Laboratory, Cambridge, Box 236, Cambridge University Hospitals NHS
Foundation Trust, Cambridge Biomedical Campus, Hills Road, Cambridge CB2 0QW, UK
uk
Received 12 December 2014
Accepted 2 April 2015
Urinalysis culminates in a workload skew within the clinical microbiology laboratory. Routine
processing involves screening via manual microscopy or biochemical dipstick measurement,
followed by culture for each sample. Despite this, as many as 80 % of specimens are reported
as negative; thus, there is vast wastage of resources and time, as well as delayed turnaround
time of results as numerous negative cultures fulfil their required incubation time. Automation
provides the potential for streamlining sample screening by efficiently (.30 % sample exclusion)
and reliably [negative predictive value (NPV) ¢95 %] ruling out those likely to be negative,
whilst also reducing resource usage and hands-on time. The present study explored this idea by
using the sediMAX automated microscopy urinalysis platform. We prospectively collected and
processed 1411 non-selected samples directly after routine laboratory processing. The results
from this study showed multiple optimum cut-off values for microscopy. However, although
optimum cut-off values permitted rule-out of 40.1 % of specimens, an associated 87.5 % NPV
was lower than the acceptable limit of 95 %. Sensitivity and specificity of leukocytes and
bacteria in determining urinary tract infection was assessed by receiver operator characteristic
curves with area under the curve values found to be 0.697 [95 % confidence interval (CI):
0.665–0.729] and 0.587 (95 % CI: 0.551–0.623), respectively. We suggested that the
sediMAX was not suitable for use as a rule-out screen prior to culture and further validation
work must be carried out before routine use of the analyser.
INTRODUCTION
Urinary tract infection (UTI) is one of the most common
indications for prescribing antibiotics. Urine samples
form a significant proportion of routine laboratory workload, but up to 80% of urine cultures are negative
(Okada et al., 2000). Therefore, a rapid and reliable
method for identifying those samples with significant bacteriuria or pyuria, and thus requiring further culture, and
those likely to be negative for UTI by culture would substantially reduce labour costs and unnecessary antimicrobial prescribing.
We sought to establish if an automated microscopy
method for urinalysis that provides bright-field pictures
of an aliquot (sediMAX; A. Menarini Diagnostics) could
be used routinely to rapidly exclude negative samples that
Abbreviations: AUC, area under the curve; CI, confidence interval; NPV,
negative predictive value; ROC, receiver operating characteristic; UTI,
urinary tract infection; WBC, white blood cell
000064 G 2015 The Authors
do not require further culture. The outcome was to
attain an acceptable negative predictive value (NPV) of
i95%, as recommended by the supplying company and
cited by Kellogg et al. (1987), and reliably exclude a proportion (w30%, determined locally for financial viability
of analyser implementation) of samples from culture techniques in order to reduce turnaround times of results. This
study aimed to formulate optimum cut-off values for
microscopy using the sediMAX automated analyser, to
produce a high NPV for UTI using the Uri System (Mast
Group) culture method, whilst maintaining adequate standards of sensitivity and specificity.
METHODS
Setting. Peterborough and Stamford Hospitals NHS Foundation
Trust is a district general hospital with 612 beds covering a population
of 450 000 people in Cambridgeshire, UK with *75 000 admissions
per year. The microbiology laboratory processed 460 502 samples in
2012, of which 78 980 (17.2%) were urine samples.
Downloaded from www.microbiologyresearch.org by
IP: 88.99.165.207
On: Fri, 16 Jun 2017 11:15:50
Printed in Great Britain
605
R. E. Sterry-Blunt and others
Data collection. Data were collected for all urine specimens submitted prospectively over a 2 week period. The sample population
incorporated general practice, inpatient, outpatient and day clinics to
ensure a representative dataset.
1.0
0.8
Specimen collection. Samples were collected in non-preservative
Specimen processing. Specimens were routinely processed to
standard protocol, consisting of a dipstick technique (Clinitek; Siemens Diagnostics) semi-automated reader and sterile aliquoting for
culture onto chromogenic, Gram-positive selective agar (Mast Group)
and positive control medium using the Uri System (Mast Group)
method. Screening was then performed with the sediMAX system for
automated microscopy analysis within 15 min following inoculation
for culture. On-screen review and necessary editing was only performed for samples which were flagged on the system as ‘crowded’, in
line with how the analyser would be used if implemented routinely.
After 15–21 h of incubation at 37 uC, plates were read by a specialist
biomedical scientist who was denied access to the results of the
sediMAX. Culture results were deemed positive if i104 c.f.u. ml21
pure growth of an organism was observed. For the purposes of this
study, a sample was considered negative for UTI if there was no
growth, no significant growth (j104 c.f.u. ml21 growth) or mixed
growth detected (two or more predominating organisms – on the
basis that this indicated contamination of the sample).
Statistical analysis. SediMAX and culture results were collated in
Microsoft Office Excel 2007 and divided into two groups determined
by the gold standard ‘culture-positive’ or ‘culture-negative’. Receiver
operating characteristic (ROC) curves were constructed for analysis of
sensitivity and specificity for leukocyte and bacterial cell components.
Curves were constructed using statistical package SPSS (IBM) in order
to determine the accuracy of the investigated parameters in predicting
UTI-positive samples.
Ethical approval. Ethical approval was not sought as this study was
deemed a service improvement. Patient management was not affected
and no patient-identifiable data were kept.
RESULTS
We processed 1411 samples over a 2 week period. The ages
of patients ranged from v1 to 99 years (median 58 years)
with 987 (70.0%) samples from females, 406 (28.8%) from
males and ‘unclassified gender’ in 18 (1.2%) cases. A total
of 387 (27.4%) urine samples tested were culture-positive
for single isolates, 22.0% of samples were classified as
‘mixed growth’ (n5311), 7.7% ‘no significant growth’
(n5109) and 42.8% ‘no growth’ (n5604). The urine culture positivity rate was 31.3, 18.2 and 22.2% in female,
male and unclassified gender groups, respectively.
Escherichia coli was the most frequent isolate (n5261;
77.4%) followed by Enterococcus faecalis (n549; 12.7%),
Klebsiella spp. (n534; 8.8%), Proteus spp. (n511; 2.8%),
Candida spp. (n510; 2.6%), Pseudomonas aeruginosa
(n59; 2.3%), coagulase-negative staphylococci (n54;
1%), group G b-haemolytic streptococci (n53; 0.8%),
Enterobacter spp. (n54; 1%) and Staphylococcus saprophyticus (n52; 0.5%).
606
WBC
Sensitivity
sterile universal containers and tested within 4 h of arrival in the
laboratory or refrigerated until processing was commenced – in accordance with the local laboratory protocol for the processing of urine
specimens.
0.6
Reference line
BAC
0.6
0.2
0
0
0.2
0.4
0.6
0.8
1.0
1 – Specificity
Fig. 1. ROC curves for WBC and bacterial cell (BAC) detection
of the sediMAX analyser demonstrating sensitivity and specificity
in prediction of UTI based upon culture result (positive or negative). Generation of WBC and bacterial cell performance characteristics: AUC50.697 (95% CI: 0.665–0.729) and AUC50.587
(95% CI: 0.551–0.623), respectively.
The ROC curves (Fig. 1) indicated the test performance for
predicting UTI based on culture result. The area under the
curve (AUC) for white blood cells (WBCs) and bacteria
was 0.697 [95% confidence interval (CI): 0.665–0.729]
and 0.587 (95% CI: 0.551–0.623), respectively.
Extensive functions composed in Microsoft Excel 2007
determined parameters of NPV, positive predictive value,
sensitivity, specificity and percentage of samples potentially
ruled out prior to culture from input combined cut-off
values for leukocyte and bacterial cell counts. Table 1 illustrates the determination of combined cut-off values for the
generation of optimum test parameters. Multiple cut-off
points generated the same level of NPV. Upon slight leukocyte adjustment (e), it was shown that subsequent parameters were the same across all optimum bacterial
counts highlighted in (d). Thus, there were multiple optimum cut-off values for the sediMAX as a screen based
upon the present data. The optimum cut-off values permitted 40.1% of specimens to be excluded from culture;
however, the associated NPV of 87.5% fell short of the
acceptable limit of 95%.
Calculation of cut-off values based upon a favourable NPV
produced no single optimal threshold criteria. Bacterial
counts of 167, 168, 169, 170, 171 and 172 bacteria ml21
gave the same optimal NPV of 87.5% when combined
with leukocyte counts of 8 and 11 leukocytes ml21. At a
threshold of 11 leukocytes ml21, percentage rule-out
Downloaded from www.microbiologyresearch.org by
IP: 88.99.165.207
On: Fri, 16 Jun 2017 11:15:50
Journal of Medical Microbiology 64
Screening urine samples for the absence of UTI
Table 1. Calculation of optimum cut-off values by NPV for leukocyte and bacterial counts (count ml21) whilst achieving .30 %
rule-out of specimens
Calculation
(a) Supplier recommendation
(b) Determined optimal leukocyte count
for local data
(c) Determining optimal range for
bacterial count
(d) Established optimal bacterial count
Bacteria cut-off
(count ml21)
Leukocyte cut-off
(count ml21)
NPV (%)
Samples ruled
out (%)
150
150
35
10
85.6
87.2
50.8
38.7
110
10
86.9
36.7
120
130
140
160
170
180
190
165
166
167
168
169
170
171
172
173
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
7
87.1
87.3
87.2
87.3
87.4
87.3
87.2
87.3
87.4
87.4
87.4
87.4
87.4
87.4
87.4
87.3
87.1
37.3
37.9
38.3
39.0
39.3
39.6
40.3
39.1
39.3
39.3
39.3
39.3
39.3
39.3
39.3
39.5
35.1
8
9
11
12
1
87.5
87.4
87.5
87.2
89.1
37.5
38.1
40.1
41.6
12.4
(e) Established optimal leukocyte count
from the range of bacteria
selected above
(f) Highest NPV achievable
241–246
(a) Parameters for recommended cut-off values by Menarini Diagnostics. (b) Optimum NPV based upon alteration of leukocyte count from the
suggested value in (a). (c) Estimating optimal range of bacterial count in terms of NPV. (d) Establishment of exact bacterial optimum cut-off. (e)
Fine adjustment of leukocyte count to determine final optimal cut-off based upon the established optimum bacterial count. (f) Highest NPV achievable from the dataset. ‘Optimum’ is highlighted in bold.
increased by 2.6% compared with 8 leukocytes ml21; thus we
determined the optimum combined cut-off values to be 11
leukocytes ml21 and 167–172 bacteria ml21, which generated
a NPV of 87.5% whilst excluding 40.1% of samples.
Although this cut-off exceeded the 30% minimal target for
sample exclusion, the NPV was v95%. The manufacturer’s
recommended cut-offs to achieve a NPV of w95% were 150
bacteria ml21 and 35 leukocytes ml21. When these parameters were applied to our data, 50.8% of samples could
be excluded, but the NPV we achieved was only 85.6%.
DISCUSSION
Our analysis suggests the platform failed the minimal targets of 95% NPV and 30% exclusion for the sediMAX to
be reliably and efficiently installed as a urinalysis screen
for negative samples. The highest NPV we achieved was
http://jmm.sgmjournals.org
89.1%, based on a cut-off of 1 leukocyte ml21, which
excluded just 12.4% of samples. Microscopy did not distinguish between positive and negative samples to an
acceptable degree for its intended use.
Our study contrasts with another study using the sediMAX
system (Falbo et al., 2012). Falbo et al. (2012) achieved a
NPV of 99.4%, but their positive culture rate was only
18%; this may have been due to their definition of UTI
(w105 c.f.u. ml21). Karakukcu et al. (2012) achieved a
NPV of 100% using another automated urinalysis system
when 85 bacteria ml21 and 13 WBCs ml21 counts were
chosen as the cut-offs. They used the same definition of
UTI as our study.
Whilst the detection of bacilli using the sediMAX was
found to be almost always correct, that of cocci presents
problems when excessive amounts of cellular or
Downloaded from www.microbiologyresearch.org by
IP: 88.99.165.207
On: Fri, 16 Jun 2017 11:15:50
607
R. E. Sterry-Blunt and others
amorphous debris are present (Zaman et al., 2010). This
may account for poor correlation between positive
microscopy and positive cultures of Enterococcus faecalis
of this study; 59.2% of Enterococcus faecalis cultures were
detected compared with 86.7% for E. coli using the established optimum cut-off values. Automated detection of
Gram-positive bacteria using microscopy may be further
compromised due to the aggregation of cells and their
small size relative to that of the more common Gram-negative uropathogens, thus increasing the false-negative
margin. This may help explain the discrepancy between
our study and others (Falbo et al., 2012; Karakukcu et al.,
2012); whilst the proportion of patients with E. coli was
similar (*60–66%), our study had a higher proportion
of Enterococcus faecalis (12 versus 7%). Whilst correction
of automated particle detection can be performed on
screen, it was only performed on those flagged as
‘crowded’. If applied for every sample, false-negative and
false-positive rates may improve. However, it should be
noted that it is felt that the resolution of the images
needs to improve.
We did not feel confidence could be placed in the detection
of certain components using image review. This is
described in other studies (Zaman et al., 2010) that
found intra-batch precision (coefficient of variation) for
erythrocytes and leukocytes to vary between 5 and 19%
over a large range of counts. Furthermore, Zaman et al.
(2010) found leukocyte and bacterial cell NPV to be 78
and 73%, respectively. Whilst it has been proposed that
sediment analysis is equivalent to strip analysis (Lammers
et al., 2001), Zaman et al. (2010) suggested that the sediMAX may only be used as a screen for negative samples
in conjunction with the dipstick method. However, such
a combination of methods defeats the purpose of installing
an automated screening method on a local level as it would
counteract a potential reduction in workload and cost.
Despite this, it supports the predilection from the present
study that the sediMAX as a stand-alone screen is not a
viable replacement for the current routine method.
Inter-laboratory disagreement exists as to what constitutes
the ‘gold standard’ culture result. Some authors believe a
culture to be negative until a count of i105 c.f.u. ml21 is
reached (Manoni et al., 2002; Falbo et al., 2012), whilst
others deem samples positive at 103 c.f.u.ml21 (Brilha
et al., 2010; Kim et al., 2007). Culture classification has previously been suggested to have a profound effect on the calculated performance of microscopy analysers. This was
demonstrated by Broeren et al. (2011), who investigated
automated urinalysis screening using the UF-1000i against
different gold standard criteria for defining negative cultures: no growth, v104 c.f.u. ml21 and v105 c.f.u. ml21.
They found optimum performance in terms of specificity
and sensitivity of the analyser was for those cultures
deemed positive at a minimum of 105 c.f.u. ml21, with leukocytes being least indicative of culture. Such criteria are
locally complicated further by separate criteria for Grampositive and Gram-negative bacteria (Koken et al., 2002),
608
mixed growth being deemed at a minimum of three organisms (Parta et al., 2013), as well as influence from clinical
input (Sultana et al., 2001). Indeed, mixed culture prevalence of this study was recorded relatively high at 22% of
total samples. It would be useful to review the origin of
these specimens as this might indicate an issue with the
quality control of sample collection. Variation in protocols
for antimicrobial prescription prior to laboratory testing
(Broeren et al., 2011) may further act to distinguish negative from positive samples and it would be useful to know
the origins of those samples recorded as ‘no growth’ on
culture. Thus, these factors may in part account for the
variation in evaluations for such analysers.
AUC values (Fig. 1) for both leukocyte (0.687; 95% CI:
0.665–0.729) and bacterial count (0.587; 95% CI: 0.551–
0.623) showed leukocytes were a better predictor of culture
by sensitivity and specificity than bacterial cell count. However, neither parameter generated sensitivity or specificity
within acceptable limits for a diagnostically useful test
(AUCi0.9). Leukocyte specificity is expected to be low
due to the range of factors capable of inducing pyuria
other than UTI. However, a high bacterial count would
also be picked up for mixed growth cultures (22.0%,
n5311) which are subsequently classified as ‘non-indicative’ of UTI, thus generating low specificity for bacterial
microscopy. The same applies for patients who have
received prior antibiotic therapy, potentially rendering bacteria non-viable. These factors may account for the finding
that bacterial testing becomes a negative predictor of UTI
towards the top right area of the curve as the ROC falls
below the line of identity (AUC50.5), representing a
non-discriminatory test. Sensitivity may be low for both
parameters due to crowding of samples (e.g. in high
levels of amorphous debris) or due to inherent reading
errors of the detection system. Our results contrast with
those of other studies (Broeren et al., 2011; De Rosa
et al., 2010), where the authors demonstrated that at any
specificity, the corresponding sensitivity was higher for
bacteria than for leukocytes. Where the ROC curve for bacterial cells falls below the line of identification in Fig. 1, culture is predicted incorrectly more often than infection is
distinguished, potentially suggesting that ‘over-reporting’
of results may have occurred.
Limitations of the study include the single-centre nature of
the study, the short time frame (2 weeks), and the fact that
aspects such as cost and process integration were not specifically considered. In addition, the pre-analytical phase (i.e.
prior to laboratory receipt) was not monitored or controlled
as this is outside the realms of routine urinalysis at this district general hospital. The relatively high proportion of
mixed cultures (22%) may have been due to poor specimen
collection (which front-line staff are expected to advise
patients upon), or time delays prior to receipt in the laboratory or processing. It may be useful for the laboratory to
provide more detailed instructions regarding appropriate
pre-analytical handling. Carry-over of specimens was
not considered in this study; however, Zaman et al. (2010)
Downloaded from www.microbiologyresearch.org by
IP: 88.99.165.207
On: Fri, 16 Jun 2017 11:15:50
Journal of Medical Microbiology 64
Screening urine samples for the absence of UTI
concluded leukocyte and bacteria cross-contamination to be
absent in their study. All samples from patients were
included, which contrasts with some other studies
(Karakukcu et al., 2012), but is similar to others (Falbo
et al., 2012). Exclusion of certain patient populations (e.g.
paediatric patients, immunosuppressed, antenatal patients,
the elderly and renal patients, as well as division of symptomatic from non-symptomatic individuals), whose samples
would likely be cultured regardless of microscopy findings,
would also affect the performance of the test. In the
future, a more focussed approach to specific patient
groups could be taken. Indeed, it may be believed that
screening applied in such an exclusionary manner may
only be suitable for non-symptomatic, otherwise healthy
patients. It may be worthwhile investigating any benefit of
a cross-check rule between sediMAX and dipstick methods
as a sample screen prior to culture. Strengths of our study
include size [our study was as large as some other published
studies (Broeren et al., 2011; Falbo et al., 2012) and larger
than others (Karakukcu et al., 2012)], time between culture
and processing was minimal, and operators were controlled,
competent and blinded to the results.
In summary, we found that the analyser cannot be reliably
used as a stand-alone negative screen for the exclusion of
samples prior to culture due to a NPV of only 89.1% in
our population. We recommend that laboratories consider
the impact of local variations in the epidemiology of UTI
on the performance of these platforms when introducing
them into their routine clinical work.
UF1000i flow cytometer for ruling out bacterial urinary tract
infection. Clin Chim Acta 411, 1137–1142.
Falbo, R., Sala, M. R., Signorelli, S., Venturi, N., Signorini, S. &
Brambilla, P. (2012). Bacteriuria screening by automated whole-
field-image-based microscopy reduces the number of necessary
urine cultures. J Clin Microbiol 50, 1427–1429.
Karakukcu, C., Kayman, T., Ozturk, A. & Torun, Y. A. (2012). Analytic
performance of bacteriuria and leukocyturia obtained by UriSed in
culture positive urinary tract infections. Clin Lab 58, 107–111.
Kellogg, J. A., Manzella, J. P., Shaffer, S. N. & Schwartz, B. B. (1987).
Clinical relevance of culture versus screens for the detection of
microbial pathogens in urine specimens. Am J Med 83, 739–745.
Kim, S. Y., Kim, Y. J., Lee, S. M., Hwang, S. H., Kim, H. H., Son, H. C. &
Lee, E. Y. (2007). Evaluation of the Sysmex UF-100 urine cell analyzer
as a screening test to reduce the need for urine cultures for
community-acquired urinary tract infection. Am J Clin Pathol 128,
922–925.
Koken, T., Aktepe, O. C., Serteser, M., Samli, M., Kahraman, A. &
Dogan, N. (2002). Determination of cut-off values for leucocytes
and bacteria for urine flow cytometer (UF-100) in urinary tract
infections. Int Urol Nephrol 34, 175–178.
Lammers, R. L., Gibson, S., Kovacs, D., Sears, W. & Strachan, G.
(2001). Comparison of test characteristics of urine dipstick and
urinalysis at various test cutoff points. Ann Emerg Med 38, 505–512.
Manoni, F., Valverde, S., Antico, F., Salvadego, M. M., Giacomini, A. &
Gessoni, G. (2002). Field evaluation of a second-generation
cytometer UF-100 in diagnosis of acute urinary tract infections in
adult patients. Clin Microbiol Infect 8, 662–668.
Okada, H., Sakai, Y., Miyazaki, S., Arakawa, S., Hamaguchi, Y. &
Kamidono, S. (2000). Detection of significant bacteriuria by
automated urinalysis using flow cytometry. J Clin Microbiol 38,
2870–2872.
Parta, M., Hudson, B. Y., Le, T. P., Ittmann, M., Musher, D. M. &
Stager, C. (2013). IRIS iQ200 workstation as a screen for
REFERENCES
Brilha, S., Proença, H., Cristino, J. M. & Hänscheid, T. (2010). Use of
performing urine culture. Diagn Microbiol Infect Dis 75, 5–8.
flow cytometry (Sysmex UF-100) to screen for positive urine cultures:
in search for the ideal cut-off. Clin Chem Lab Med 48, 289–292.
Sultana, R. V., Zalstein, S., Cameron, P. & Campbell, D. (2001).
Broeren, M. A. C., Bahçeci, S., Vader, H. L. & Arents, N. L. A. (2011).
Dipstick urinalysis and the accuracy of the clinical diagnosis of
urinary tract infection. J Emerg Med 20, 13–19.
Screening for urinary tract infection with the Sysmex UF-1000i urine
flow cytometer. J Clin Microbiol 49, 1025–1029.
Zaman, Z., Fogazzi, G. B., Garigali, G., Croci, M. D., Bayer, G. &
Kránicz, T. (2010). Urine sediment analysis: analytical and
De Rosa, R., Grosso, S., Bruschetta, G., Avolio, M., Stano, P.,
Modolo, M. L. & Camporese, A. (2010). Evaluation of the Sysmex
diagnostic performance of sediMAX – a new automated microscopy
image-based urine sediment analyser. Clin Chim Acta 411, 147–154.
http://jmm.sgmjournals.org
Downloaded from www.microbiologyresearch.org by
IP: 88.99.165.207
On: Fri, 16 Jun 2017 11:15:50
609