Family Practice 2013; 30:325–331
doi:10.1093/fampra/cms076
© The Author 2012. Published by Oxford University Press. All rights reserved.
For permissions, please e-mail: [email protected].
Advance Access publication 11 December 2012
Using qualitative research to inform development of a
diagnostic algorithm for UTI in children
Isabel de Salis*, Penny Whiting, Jonathan A C Sterne and Alastair D Hay
School of Social and Community Medicine, University of Bristol, Bristol, UK.
*Correspondence to Isabel de Salis, School of Social and Community Medicine, University of Bristol, Canynge Hall, 39 Whatley
Road, Bristol BS8 2PR, UK; E-mail: [email protected]
Received 18 May 2012; Revised 22 October 2012; Accepted 10 November 2012.
Background. Diagnostic and prognostic algorithms can help reduce clinical uncertainty. The selection of candidate symptoms and signs to be measured in Case Report Forms (CRFs) for potential
inclusion in diagnostic algorithms needs to be comprehensive, clearly formulated and relevant for
end users.
Objective. To investigate whether qualitative methods could assist in designing CRFs in research
developing diagnostic algorithms. Specifically, the study sought to establish whether qualitative
methods could have assisted in designing the CRF for the Health Technology Association funded
Diagnosis of Urinary Tract infection in Young children (DUTY) study, which will develop a diagnostic algorithm to improve recognition of urinary tract infection (UTI) in children aged <5 years
presenting acutely unwell to primary care.
Methods. Qualitative methods were applied using semi-structured interviews of 30 UK doctors
and nurses working with young children in primary care and a Children’s Emergency Department.
We elicited features that clinicians believed useful in diagnosing UTI and compared these for presence or absence and terminology with the DUTY CRF.
Results. Despite much agreement between clinicians’ accounts and the DUTY CRFs, we identified
a small number of potentially important symptoms and signs not included in the CRF and some
included items that could have been reworded to improve understanding and final data analysis.
Conclusions. This study uniquely demonstrates the role of qualitative methods in the design and
content of CRFs used for developing diagnostic (and prognostic) algorithms. Research groups
developing such algorithms should consider using qualitative methods to inform the selection and
wording of candidate symptoms and signs.
Keywords. Algorithm, methodological study, qualitative research, urinary tract infections.
in the design of the CRF could ensure that relevant
(and contentious features) arising within daily practice, probably unbeknown to experts in the field, are
covered in the CRF, thereby improving an algorithm’s
uptake and its ultimate impact. To our knowledge, formal qualitative research methods have not been used to
inform the CRF in diagnostic or prognostic algorithm
development.
We therefore conducted a retrospective investigation
to see how qualitative methods could have assisted the
design of a CRF for the Diagnosis of Urinary Tract
infection in Young children (DUTY) study,2 which will
develop a diagnostic algorithm to improve recognition of
urinary tract infection (UTI)3 in children aged <5 years.
Our objective was to identify the clinical features that
clinicians believe to be useful in diagnosing UTIs and
to compare these with the symptoms and signs included
in the CRF.
Introduction
There is growing interest among researchers and clinicians regarding the potential for diagnostic and prognostic algorithms to reduce clinical uncertainty. Study
groups developing diagnostic and prognostic algorithms
first need to decide the symptoms and signs that could
be relevant for their algorithm and how they should be
measured in the Case Report Form (CRF). Selection
of candidate symptoms and signs are generally determined through previous evidence from the literature,
research team debate and consensus among experts
leading the research. However, such processes alone
may not lead to comprehensive coverage of important items and selected symptom or sign terminology
may not always be recognizable to patients and carers. Furthermore, uptake of algorithms in clinical practice is known to be poor.1 Using a range of clinicians
325
326
Family Practice—The International Journal for Research in Primary Care
Methods
Semi-structured interviews were conducted by the
first author (IdeS) with clinicians responsible for the
management of acutely unwell pre-school children in
National Health Service walk-in centres, general practices and a Children’s Emergency Department (CED)
in South West and North West England. Clinicians were
recruited via contacts within the University of Bristol,
Academic Unit of Primary Health Care and the DUTY
study and they were purposively sampled to include
varied clinical settings, experience, role and research
interests. Interviews were structured around a topic
guide to ensure coverage of key issues in diagnosing
childhood UTIs: symptoms, signs, tests, diagnosis and
management. This was used flexibly to allow unanticipated themes to emerge, and clinicians were encouraged
to draw on experience rather than reproduce textbook
accounts. The study had ethics approval [South West 3
Research Ethics Committee: 09/HO106/85] and interviewees gave written informed consent for publishing
anonymized quotes. Interviews lasted between 30–40
minutes and were audio recorded and transcribed.
Transcripts were anonymized and checked for misunderstandings or transcription errors.
Analysis was thematic:4 key themes were developed and
then compared across the data set with attention to clinical role and active involvement in the DUTY study. Data
familiarization was facilitated by Nvivo software. Coding
was guided by the topic guide, original research questions
and emergent issues in the data. Areas of potential disagreement and lack of consensus were sought. Codes were
cross-checked and key themes developed and refined by
two team members (IdeS and ADH). Emergent themes
were pursued in subsequent interviews, some developing
into key themes and others losing importance. Towards
completion of data collection, findings were discussed
with the whole team. Clinical features were ranked
according to number of clinicians reporting them and
their accredited importance in interview. Credence was
given to all features regardless of how often mentioned
but considering clinicians’ level of experience.
Findings were compared with the content of the
DUTY CRF. The DUTY study, which began in 2010 in
four centres in England and Wales, recruited acutely ill
children presenting to primary and emergency care who
were systemically unwell for <28 days. The CRF (see
Supplementary material) was completed by the recruiting nurse or officer and the clinician responsible and
consisted of five sections: eligibility criteria; participant
registration; severity of the condition, symptoms and
medical history; results of physical examination, working diagnosis and planned management; and method of
urine collection and dipstick results.
Results
We interviewed 30 clinicians working in nine inner-city
practices serving a range of socio-demographic areas;
three walk-in centres; one suburban practice; seven town
practices (three rural, four peri-urban); and a Children’s
Emergency Department (CED). Practices served populations ranging between 2000 and 15 000 patients.
There were 10 nurse practitioners (NPs), 10 GPs, 5 CED
nurses and 5 CED doctors, all actively diagnosing UTIs
in pre-school children (Table 1). Two were interviewed
by telephone; the other 28 had face-to-face interviews.
Table 1 Characteristics of clinicians interviewed during the study
Clinical role
N
Research and training
Nurse practitioners
10
Paediatric specialist (1)
GPs
10
Research role (7)
Research role in paediatrics (1)
CED nurses
5
CED doctors
5
Staff nurses (4)
Emergency NP (1)
Paediatric ED Consultant (1)
ED Consultant (1)
Trainees (3)
Number of acutely unwell
children seen per week
Qualifying dates
Number of clinicians
involved in recruitment to
DUTY study
<5 (2)
5–9 (5)
10–15 (2)
>15 (1)
<5 (4)
5–9 (8)
10–15 (6)
>15 (2)
70–100
50–60
~50
<15
20–40
1980s (5)
1990s (5)
4
1970s (2)
1980s (10)
1990s (8)
1
0
1970s (1)
1980s (1)
1990s (2)
2000 + (6)
0
CED, Children’s Emergency Department; ED, Emergency Department; GP, General Practitioner; NP, Nurse Practitioners; DUTY, Diagnosis of
Urinary Tract infection in Young children. (N) refers to number of clinicians.
Qualitative research in development of a diagnostic algorithm for UTI
Clinicians identified a broad range of clinical signs and
symptoms as potentially useful for ruling in and ruling
out UTIs. Table 2 summarizes the clinical features identified by clinicians as helpful in diagnosing childhood
UTI compared to the DUTY CRF. Some highlighted
that several symptoms and signs often thought diagnostic for UTI may be unreliable or insufficiently specific
in young children, e.g. dysuria, vomiting, abdominal
pain and tenderness (primary care clinicians), whereas
at other times, they focused on the optimal wording
used to capture the essence of the symptom:
Everybody’s pretty aware of signs and symptoms
as an adult but with children I think, if I’ve got a bit
of tummy pain it could be from anywhere, head to
toe really… So I do think they can be difficult from
that point of view and yeah I think they do need
much more questioning. (NP5: DUTY [recruiting
to DUTY study])
The following features were identified as helpful in
suggesting a diagnosis of UTI but were not included in
the DUTY CRF. Some features were additional signs and
symptoms: history of febrile convulsions (2 clinicians),
differing presentations of dysuria in UTI (5), absence
of sore genitalia (4), decreased urinary frequency (4),
not drinking fluids (6), absence of history of infectious
contacts (4), change in urine colour (3) and back
pain (3). Other features were qualifiers to existing
signs and symptoms: fluctuating fever (3), parental
measurement of fever (4), direct elicitation of symptoms
disproportionate to the attributed cause (6), persistent
dysuria (1), urinary appearance assessed by clinician
(2), persistent vomiting (3), cross-checking parental
meaning of terms ‘vomiting’ (1) and ‘diarrhoea’ (2).
Further elaboration is provided in the following sections.
Fever: presentation and measurement
Clinicians noted that they thought UTI fever was often
high and fluctuating (especially at night) and can be
accompanied by convulsions.
So a fever that peaks, troughs over a couple of
days…Parents have said well you know, there’s a
fever, then it went down and then today it’s gone
up… I just think that as a whole really, leads you to
think of (UTI). (NP2: non-DUTY [not recruiting
to DUTY study]).
They reported carefully questioning parents about
the nature and severity of fever and how it had been
assessed:
But you might not get a fever at the time you actually saw the child so you have to listen to the parents carefully on that one. We could talk about
fever quite a lot if you like because there’s quite a
problem here because most people don’t seem to
327
have thermometers….And so people assume that
if the child feels hot they have a fever, and that’s
often the case, but... So we’re talking about taking a careful history and I am quite careful. (GP 3:
non-DUTY)
Symptoms disproportionate to the attributed cause
Clinicians noted that a typical UTI presentation was
degree of fever or unwellness being greater than the
attributed cause.
And then there’s these things that just come with
experience if a child looks ill. Disproportionately
ill. Again, the problem with upper respiratory tract
infection, which most of these are, is that you can
almost persuade yourself that any child’s got an
upper respiratory tract infection, a bit of a pink ear
or a bit of a red throat. (GP4: non-DUTY)
Say the parents were saying, ‘Oh he’s got a really
snotty nose but actually the fever seems higher’,
I would dip the urine then. (NP6: DUTY)
Dysuria for ruling in and ruling out UTI
UTI in younger children may not present as classic dysuria but by a child wanting to urinate in the bath, crying
followed by a wet nappy or vaginal itching. Ensuring
the persistency of dysuria is helpful for ruling in UTI. In
young children, the symptom of dysuria may be caused
by concentrated urine or sore genitalia:
But one of the things I learnt at outpatients is that
I think the dysuria and frequency especially in the
little girls and in boys with balanitis can be because
they’re sore, it is not because they’ve got cystitis it is
because their vulva is sore. (CED DR5: non-DUTY)
It’s important to ask how often, are they getting pain
every time, or is it just a one-off, or and also to have a
look and see if there’s anything there, which for this
child it was, it was a small tear so it wasn’t a urinary
tract infection. And if I hadn’t examined her I would
have missed that, or if I hadn’t got more of a history.
So mum saying about the stinging on passing urine,
that kind of made me think well maybe she’s right, but
actually on a bit more questioning it wasn’t, and on
examination…I wouldn’t always examine the genital
area, so maybe that’s worth doing. (NP1: non-DUTY)
Urine appearance and reduced urinary frequency and
fluid intake
Some clinicians thought dry nappies could indicate
UTI if a child avoids urinating or has reduced fluid
intake. Clinicians, including all GPs, spontaneously
commented on the significance of urinary appearance and smell, with doctors often disputing its
significance:
328
Family Practice—The International Journal for Research in Primary Care
Table 2 Clinical features identified as helpful in diagnosing childhood UTI compared to DUTY CRF
Clinical feature
Fever
General fever
Prolonged fever, i.e. >2–3 days
Fluctuating (especially at night)
Checked for history of fever
If yes, checked how measured
If used thermometer, asked reading
Checked if analgesics used
History of rigors (shaking, shivering)
History of febrile convulsion
UTI possible without fever
>38° on examination
Unwellness
General unwellness (flat out, miserable, lethargic, irritable, grizzly, bit floppy, hanging
about, listless, tearful, whinging, drowsy, unhappy, clingy, not themselves, off colour,
unsettled, not talking, in pain)
Chronicity >few days
Repeat consultations/previously consulted GP
More unwell than when normally ill/potentially toxic or septic
Not feeding
Not drinking fluids
Failure to thrive
Not sleeping well
History of change in colour, pale
Examination of colour (pale, mottled, flushed, clammy, change)
Dehydrated on examination
Symptoms disproportionate to the attributed cause
Examination: not severe
Urinary symptoms/signs
Dysuria
Alternative presentations of dysuria in UTI (e.g. child wanting to urinate in the
bath, crying followed by a wet nappy, vaginal itching)
Persistent (>once)
Frequency increased
Frequency decreased
Newly incontinent/bed wetting
History of blood in urine
History of dark, offensive urine
Check in clinic
Heuristic/rule of thumb
Fever, no focus/cause, think UTI
Vomiting
Vomiting
Persistent vomiting/frequency
Cross-checking parental understanding of term
Without diarrhoea
Abdominal symptoms/signs
Abdominal pain
And back pain
Constipation
Diarrhoea
Cross-checking parental understanding of term
Abdominal tenderness
And back/loin/suprapubic tenderness
Past medical history
Renal disease, abnormality
History of UTIs
Symptoms/signs to exclude UTI
History of infectious contacts (gastroenteritis and other viral)
Clinical role (GP, NP,
CEDDR, CEDN)
Included in CRF
All
8, 2, 1, 1
2, 1, 0, 0
3, 0, 0, 0
3, 1, 0, 0
3, 1, 0, 0
1, 2, 0, 0
2, 1, 2, 0
1, 0, 1, 0
3, 0, 1, 2
6, 6, 1, 1
Yes
Yes
No
Yes
No
No
Yes
Yes
No
Yes
Yes
10, 6, 4, 5
Yes
6, 4, 1, 3
4, 1, 0, 0
6, 2, 3, 3
7, 5, 3, 1
1, 4, 0, 1
1, 0, 0, 0
0, 1, 0, 0
1, 2, 0, 0
3, 3, 0, 2
3, 4, 4, 2
5, 1, 0,0
0, 0, 2, 1
Yes
Yes
Yes
Yes
No
Yes
Yes
No
Yes
Yes
Indirect1
Yes
7, 8, 4, 5
0, 4, 0, 1
Yes
No
0, 1, 0, 0
2, 4, 1, 4
0, 3, 0, 1
7, 5, 1, 3
2, 0, 0, 2
10, 6, 4, 4
0, 1, 1, 0
No
Yes
No
Yes
Yes
Yes
No
8, 6, 4, 4
Yes
9, 4, 5, 3
2, 0, 1, 0
1, 0, 0, 0
3, 0, 2, 0
Yes
No
No
Yes
7, 7, 2, 3
1, 1, 0, 1
0, 1, 0, 0
1, 0, 1, 0
1, 0, 1, 0
8, 3, 4, 1
1, 1, 2, 0
Yes
No
Yes
Yes
No
Yes
Yes
4, 4, 0, 0
6, 6, 1, 3
Yes
Yes
3, 0, 1, 0
No
329
Qualitative research in development of a diagnostic algorithm for UTI
Table 2 Continued
Clinical feature
Clinical role (GP, NP,
CEDDR, CEDN)
Upper respiratory tract (and ear, nose, throat) infections (cough,
sneezing, red ear, ear pain, runny nose, snotty, pus in throat, cervical lymph nodes,
tonsillitis)
Lower respiratory tract infection (pneumonia, bronchiolitis.
breathlessness, rapid breathing, wheezing)
Gastroenteritis (diarrhoea and vomiting)
Influenza (headache, miserable, achy)
Meningitis (rash, limb pain, photophobia, drowsy)
Viral illness (rash)
Appendicitis/acute abdomen
Dysuria caused by local irritation/soreness, not UTI
Examination of genitalia for non-UTI dysuria
Gender effect
More common or less serious in females
Included in CRF
10,8, 3, 5
Yes
4, 2, 4, 2
Yes
6,2, 4, 3
2, 0, 0, 0
7, 1, 4, 2
2,4, 1, 1
0, 1, 1, 0
0, 3, 1,0
0, 3, 1, 0
Yes
Yes
Yes
Yes
Yes
Indirect2
No
2, 0, 1, 1
Yes
GP, General Practitioner; NP, Nurse Practitioner; CEDDR, Emergency Doctor; CEDN, Emergency Nurse; CRF, Case Report Form.
Indirect1: elicits indirectly through multiple questions; Indirect2: question on nappy rash.
No and particularly not things like cloudy or smelly
urine, because a lot of people think that means
they’ve got a UTI and it doesn’t, your urine can be
smelly because it is concentrated and I’ve seen the
cloudier specimens come back negative. So I’ve
learnt that over the years it’s often precipitants in
the urine in the crystals that makes it cloudy not
bugs. (CED DR5: non-DUTY)
Others thought it helpful, some checking it themselves:
Mum usually says urine is very dark or smelling
very strong…You look at it, see if there’s anything
in it, how dark it is, and when you test it you smell
it. (NP10: non-DUTY)
Vomiting
All clinicians considered UTI in vomiting, feverish children without diarrhoea. Some stressed careful questioning of parents to ensure definite, persistent vomiting:
But again, vomiting, I find that really non-specific
in children. They just vomit when they’ve got a
temperature. And parents mean very different
things when they say a child has vomited. Sometimes
they just mean they feel sick. Sometimes they mean
they’ve had a really nasty cough, with lots of mucous
and they’ve coughed and coughed and coughed
and then sometimes vomited. So you really have to
unpick what that means. But vomiting is something
I would attribute to the urine infection….if it’s
persistent vomiting. (GP5: non-DUTY)
Diarrhoea
Clinicians noted that mild diarrhoea (e.g. slightly loose
stools), considered possible with UTI, needed distinguishing from full-blown diarrhoea of gastroenteritis:
I mean you need to know what they (parents)
mean when they say diarrhoea. So if you say, “Have
you had diarrhoea?” you need to always ask, “How
many dirty nappies are you having in a day?” and
gastro-enteritis in babies are having 10, 12 dirty
nappies a day. I had one yesterday. So 2 loose
stools doesn’t necessarily make a gastro-enteritis.
So I would be a little bit concerned, especially if
they’re febrile, to look for something else and a
UTI would probably be high up on that list. (CED
DR4: non-DUTY)
Alternative infective explanation and infectious
contacts
Clinicians reported that alternative explanations for
the child’s symptoms or history of infectious contacts
can be helpful for ruling out UTI:
Well, upper respiratory tract infections is the biggie. That’s what they mostly would have, or other
viral infections of one sort or another….The other
biggie would be GI infections, diarrhoea, …and
that is often on the history of who else, bugs like
that are often going around, for example, another
child’s had it, or it’s in the nursery or in the schools,
that could help to support that bit of diagnosis.
(GP5: non-DUTY)
But if they don’t have any of those (respiratory)
symptoms but they’ve got a temperature then I’d
wonder if they had a rash or had been in contact
with anyone who had any other, German measles
and measles or chicken pox or something. If there is
no other clues and they are just hot and unwell and
no obvious cause, then I would think about urinary
tract infection. (GP10: non-DUTY)
330
Family Practice—The International Journal for Research in Primary Care
Discussion
Our evidence suggests there is a role for qualitative
research to support diagnostic and prognostic studies by
helping inform the content of the CRF. Although there
was much agreement between clinicians’ accounts and
the DUTY CRF, our study identified a number of additional candidate items, some of which were additional
symptoms or signs and others were qualifiers to existing
symptoms or signs: symptoms disproportionate to the
cause; fever (convulsions and intermittency, assessment
method, highest temperature recorded); dysuria (alternative presentations, history and examination of sore
genitalia); reduced urinary frequency and fluid intake;
clinician assessment of urine presentation; vomiting
and diarrhoea (duration, frequency and cross-checking
parental understanding of terms); and the absence of a
history of infectious contacts.
Clinician interviews suggested how some symptom
areas that were recorded in the CRF could have been
improved. Further questioning in the CRF on the qualities of fever and how it was assessed might have elicited useful diagnostic information. Similarly, presence
of vomiting and diarrhoea was recorded in the CRF
but clinicians suggested that more specific questioning of parents’ understanding of these terms and further symptom details were required. National Institute
for Health and Clinical Excellence (NICE) guidelines
consider vomiting helpful for diagnosing UTI,5 and
although the literature may imply otherwise,6 it is possible that previous studies did not collect sufficiently
detailed information to identify diagnostically useful
questions.
For other symptoms, clinician interviews indicated
that the CRF questions were too closed and that generic
questions may have been more informative. For example,
the CRF asks only about increased frequency, rather than
change in urinary frequency—some clinicians believed
decreased urinary frequency to be diagnostic. For other
areas covered by multiple questions in the CRF, a specific question suggested by the interviews could have
simplified its analysis. For example, the CRF asks clinicians to assess the severity of signs and state how certain
they are of their working diagnosis. However, the direct
question “Does the child appear sicker than expected
from your working diagnosis?” might have been useful.
Areas missed altogether included the following: history
of infectious contacts; reduced drinking (despite reduced
or no fluid intake being a potentially useful indicator for
UTI)6; history and possible examination of sore genitalia; clinician assessment of urinary presentation, despite
being a contentious sign that is valued by some practising
clinicians but not by leading experts in the field.
Our study focused on clinicians’ accounts, which were
sufficient to investigate whether such methods could
be useful in compiling a more comprehensive CRF.
However, if time and resources had permitted, we would
have also investigated parental perspectives and will do
so in future research. Eight clinicians were research active,
four clinicians were recruiting to the DUTY study and
using the CRF, and one clinician was a lead researcher in
DUTY. Participation in DUTY might have changed their
ideas about diagnosis, making them less likely to suggest
alternative symptoms or signs. However, these clinicians
did provide notable examples of clinical features not
mentioned in the CRF. Our sample was diverse, increasing
the likelihood of eliciting relevant information in
diagnosing UTIs in young children: clinicians had varied
roles, worked in different clinical contexts, and in practices
of different sizes and demographics. Clinicians’ practices
and knowledge, derived from extensive experience, were
explored in detail. The impact of this study would have
been greater had it been initiated in time to inform the
design of the CRF.
Incorporation of patient perspectives may also be
useful: for example, a study designed to improve diagnosis of ovarian cancer used interviews with patients
with suspected malignancy alongside quantitative
methods to identify a broad set of potentially significant diagnostic factors, using patients’ symptom terminology (which differed from medical terminology).7
Focus groups are an alternative to interviews, as suggested by Beattie and Nelson8 in their overview on
how to use clinical prediction rules, although these may
be less good for eliciting areas of dissent.9 In our study,
the strongly held differences of opinion about urine
appearance, for example, may have been less apparent. Observational data could also help to develop
a CRF in a similar way that Krebes et al.10 analysed
emergency calls of stroke patients to develop the final
algorithm itself. We suggest that qualitative methods
be used alongside expert knowledge and systematic
reviews, if present, and to inform other methods developed to select predictors for prognostic or diagnostic
algorithms, such as the Delphi technique.11,12 Whereas
the latter are primarily tools for consensus making and
ranking,13 qualitative methods such as interviews are
tools for in-depth exploratory work that can generate
new topics, which are explored in depth and understood in context.
Formal qualitative research methods, in-depth interviews in this case, may ensure a more comprehensive
final diagnostic or prognostic algorithm by including
end users’—either clinicians’ and/or patients’—perspectives in the selection of candidate predictors used
in CRFs, thereby improving the predictive performance
of the prediction rule. Such methods may also improve
the applicability of the prediction rule by ensuring predictors are relevant, clearly formulated and in sufficient
detail, as well as improving the uptake of the algorithm
in clinical practice.1 We believe that qualitative research
should be considered in selection of candidate signs
and symptoms for inclusion in CRFs for all diagnostic
and prognostic studies.
Qualitative research in development of a diagnostic algorithm for UTI
Supplementary material
Supplementary material is available at Family Practice
online.
‍Acknowledgements
The authors would like to thank the clinicians who participated in this study; all the DUTY team members,
particularly Harriet Downing and Catherine Derrick,
for their support; and Dr Alison Heawood for comments on the manuscript.
Declaration
Funding: Medical Research Council National Institute
for Health Research (MRC NIHR) Methodology
Research Programme (G0801405); and the NIHR
Health Technology Association-funded DUTY study
(08_66).
Ethical approval: South West 3 Research Ethics
Committee (09/HO106/85).
Conflicts of interest: none.
References
Bruins Slot MHE, Rutten FH, van der Heijden GJMG, et al.
Diagnosing acute coronary syndrome in primary care:
1 331
comparison of the physicians risk estimation and a clinical
decision rule. Fam Pract 2011; 28: 323–28.
2 The Diagnosis of Urinary Tract Infection in Young Children Study.
www.dutystudy.org.uk/ (accessed on 31 October 2012).
3 NICE. Urinary Tract Infection in Children: Diagnosis, Treatment
and Long-Term Management. London, UK: National Institute
for Health and Clinical Excellence; 2007.
4 Miles M, Huberman M. Qualitative Data Analysis: An Expanded
Sourcebook. London, UK: Sage; 1994.
5 NICE. CG54 Urinary Tract Infection in Children: Full Guidelines.
www.nice.org.uk/nicemedia/pdf/CG54fullguideline.pdf
(accessed on 31 October 2012).
6 Hay AD, Whiting P, Butler CC. How best to diagnose urinary
tract infection in preschool children in primary care? BMJ
2011; 343: d6316.
7 Bankhead CR, Collins C, Stokes-Lampard H, et al. Identifying
symptoms of ovarian cancer: a qualitative and quantitative
study. BJOG 2008; 115: 1008–14.
8 Beattie P, Nelson R. Clinical prediction rules: what are they and
what do they tell us? Austral J Physiother 2006; 52: 157–63.
9 Kitzinger J. Qualitative research. Introducing focus groups. BMJ
1995; 311: 299–302.
10 Krebes S, Ebinger M, Baumann AM, et al. Development and validation of a dispatcher identification algorithm for stroke emergencies. Stroke 2012; 43: 776–81.
11 Vergouw D, Heymans MW, de Vet HC, van der Windt DA, van
der Horst HE. Prediction of persistent shoulder pain in general practice: comparing clinical consensus from a Delphi
procedure with a statistical scoring system. BMC Fam Pract
2011; 12: 63.
12 Meijer R, Ihnenfeldt D, Vermeulen M, De HR, Van LJ. The use of
a modified Delphi procedure for the determination of 26 prognostic factors in the sub-acute stage of stroke. Int J Rehabil Res
2003; 26: 265–70.
13 Jones J, Hunter D. Consensus methods for medical and health services research. BMJ 1995; 311: 376–80.