Quality of Life Research 10: 59±70, 2001.
Ó 2001 Kluwer Academic Publishers. Printed in the Netherlands.
59
Elementary school-aged children's reports of their health: A cognitive
interviewing study
G. Rebok1, A. Riley2, C. Forrest2, B. Star®eld2, B. Green3, J. Robertson2 & E. Tambor2
1
Department of Mental Hygiene (E-mail: [email protected]); 2Department of Health Policy and
Management, School of Public Health; 3Department of Psychology, School of Arts and Sciences, The Johns
Hopkins University, Baltimore, USA
Accepted in revised form 13 February 2001
Abstract
There are no standard methods for assessing the quality of young children's perceptions of their health and
well-being and their ability to comprehend the tasks involved in reporting their health. This research
involved three cross-sectional studies using cognitive interviews of 5±11-year-old children (N = 114) to
determine their ability to respond to various presentations of pictorially illustrated questions about their
health. The samples had a predominance of children in the 5±7-year-old range and families of lower and
middle socio-economic status. The research questions in Study 1 involved children's ability to convert their
health experiences into scaled responses and relate them to illustrated items (n=35); Study 2 focused on the
type of response format most e€ectively used by children (n=19); and Study 3 involved testing children's
understanding of health-related terms and use of a speci®c recall period (n=60). The results of Study 1
showed that children identi®ed with the cartoon drawing of a child depicted in the illustrated items,
typically responding that the child was at or near their own age and of the same gender, with no di€erences
related to race. Study 2 results indicated that children responded e€ectively to circles of graduated sizes to
indicate their response and preferred them to same-size circles or a visual analogue scale. Tests of three-,
four-, and ®ve-point response formats demonstrated that children could use them all without confusion. In
Study 3, expected age-related di€erences in understanding were obtained. In fact, the 5-year-old children
were unable to understand a sucient number of items to adequately describe their health. Virtually all
children 8 years of age and older were able to fully understand the key terms and presentation of items,
used the full ®ve-point range of response options, and accurately used a 4-week recall period. Six- and
seven-year-olds were more likely than older children to use only the extreme and middle responses on a ®vepoint scale. No pattern of gender di€erences in understanding or in use of response options was found. We
conclude that children as young as eight are able to report on all aspects of their health experiences and can
use a ®ve-point response format. Children aged 6±7 had diculty with some health-related terms and
tended to use extreme responses, but they understood the basic task requirements and were able to report
on their health experiences. These results provide the guidance needed to develop and test a pediatric health
status questionnaire for children 6±11 years old.
Key words: Child health status, Cognitive interviewing, Health assessment, Response format
Introduction
Children have a unique perspective on their own
health, and may be able to provide invaluable information to health care professionals, health
planners, and health policy makers. Obtaining
this information via children's self-reports seems
increasingly possible [1]. However, there are no
existing instruments/tools to capture their expressions of health and well-being in a systematic
60
manner [2]. This has very practical implications
since neither parents nor clinicians are always able
to adequately report on children's internal health
experiences. Being able to assess children's perceptions of their well-being and their experience of
somatic and emotional symptoms is critical to
managing their health as well as to understanding
how these experiences may in¯uence their achievement of developmental tasks and everyday functioning [3, 4]. The challenge, then, is to provide
children themselves with a means for describing
the important aspects of their physical and emotional well-being.
The studies reported here are the ®rst in a project to develop a generic pediatric health questionnaire for children aged 5±11 years. A better
scienti®c basis was needed to determine if the
youngest children could report on their own
health, and if so, how to design an instrument
to maximize the reliability and validity of their
responses.
Regardless of age, in order to complete a health
survey, a person must have at least a rudimentary
self-concept, understand the basic notions of
health and illness, be able to pay attention, comprehend the questions, discriminate between the
response alternatives, recall health experiences,
and write a response. Prior to this research, the
assumption was that children under the age of 9
or 10 years could not be reliable or valid reporters
of general health status. Consequently, parents
or caregivers are the expected respondent on every
questionnaire about the health of children up to
10 years of age [2]. Indeed, it is rarely even recognized that parents are `proxy respondents' for
their children.
Drawing on the developmental literature on
children's abilities, four objectives were developed
for the research. First, using cognitive interviewing
methodology, we sought to determine whether
children in the targeted age range can answer
health survey items. The cognitive interview provided speci®c sets of questions, response options,
and health-related terms within a de®ned interviewer-based assessment protocol. Second, since
many health survey instruments are rather lengthy
and draw on concepts that may be too abstract for
younger children, we tested the feasibility of a
pictorial questionnaire format using cartoon
drawings of a `universal' child to illustrate key
concepts. Illustrations have been used successfully
in a number of child questionnaires [5±7] and have
the advantages of maximizing attention to the task
and minimizing reliance on younger children's
limited vocabularies. Third, we examined several
types and numbers of response formats to see
which are most easily understood by young children and which they prefer. Finally, we tested
children's understanding of speci®c concepts of
health and wording of the di€erent response
formats.
Literature review
It is known that even children as young as
5 years old can describe internal mental states
such as perceptions, emotions, cognitions, and
physiological states, but there was concern that
they are unable until about the age of seven or
eight to distinguish between their inner experience
and the external behavior that others see [5]. We
were unsure whether young children would be
able to distinguish between di€erent aspects of
themselves (good at numbers, but poor at reading), expecting that they would show evidence of
`all or none thinking' until at least 8 years of age
[8±10].
Language mastery is likely to limit young children's ability to describe their health. There is
limited information about the breadth of a typical
young child's health vocabulary or comprehension
of health questions. The literature demonstrates
that even young children can respond to questions
about pain [11±13] and nausea [14], with 5- and
6-year-old children providing more variable and
less discriminating responses than older children.
It is also clear that children's understanding of
health-related words, ability to understand complex sentences, and to comprehend and match
verbally presented sentences with illustrations
increase with age [8, 15] and that 5-year-olds are
likely to incorrectly use relational terms (more/
less, same/di€erent) [16].
In terms of their concept of health, children
below age eight were expected to view health as a
series of speci®c health practices and were not
expected to understand that they could be partially
healthy and partially not healthy [17, 18]. We did
not expect children below age eight to understand
that illness is de®ned by a set of concrete symp-
61
toms, or to use internal cues to identify the presence of illness [4, 10, 19, 20]. Based on the literature children in our entire age spectrum were not
expected to be able to think about future health or
to have a concept of mental health [17, 18].
The ability of children to sustain attention
increases rapidly between the ages of 5 and 7 years
[21±23] as does their ability to inhibit irrelevant
responses [23, 24]. Even at 4 years of age, children
show high accuracy in discriminating among faces
depicting di€erent emotions and matching one
that `feels the same' to a standard [25]. Further,
they steadily increase in their ability to discriminate and match symbols [26, 27].
In terms of recall, 5-year-olds are able to accurately recall routine and novel events for at least
24 hour [28, 29] and can recall novel events for
weeks by age seven [28, 30]. However, not until age
seven or older can children reason about the timing of past events in terms of the day of the week,
month, or season [31]. No research was found on
children's use of di€erent recall periods.
The literature suggested that visual analogue
scales (VASs) provided a method that children
®nd engaging and can understand, at least for
reports of pain [11, 32]. Most of this work has
been in clinical experiments in medical settings
where children experience and report on the intensity of their pain. In contrast, most questions
in population-focused health surveys are about
the frequency or recency of health and illness
experiences. Illustrated items also have been
successfully employed in questionnaires for children. However, no empirical work exists on the
e€ects of the age, gender, or race of the illustrated character on the quality of children's responses. Most notable is a questionnaire
developed by Valla and colleagues, which they
call `Dominic.' Dominic is an intervieweradministered assessment in which children aged
6±11 years respond to illustrated items about the
presence or absence of psychiatric symptoms [6,
7, 33]. The 95 illustrated items produce seven
psychiatric diagnoses with test±retest reliability
comparable to that obtained by much longer
interviews of older children and adolescents 9±17
years old [33]. The Violence Exposure Scale for
Children-Revised [34, 35] is designed for elementary aged (6±8 years) children and uses a set
of four thermometers to identify the frequency
with which the respondent child has experienced
each type of violence. It also has been shown to
produce reliable data from preschool and kindergarten children [34].
Thus, the literature supported the feasibility of
developing a health questionnaire for children.
However, there were signi®cant gaps in understanding about: (1) the optimal ways to ask children questions about their health; (2) preferences
for item presentation and response formats; (3)
ability to handle up to ®ve response options and
speci®c recall periods; (4) knowledge of healthrelated terms; and (5) ability to report on aspects
of their health other than pain and emotional
symptoms. These issues guided the experimental
questions posed in this cognitive interviewing study.
The content of the questions drew from earlier work conducted by the investigators on the
Child Health and Illness Pro®le-Adolescent Edition (CHIP-AE) [36±38]. The CHIP-AE is a selfadministered health status measure for use with
adolescents aged 11 through 17 years. The measure
is comprised of six domains of health (satisfaction
with health, discomfort, risks, resilience, disorders,
and achievement) and 20 subdomains that were
conceptually derived and supported by factor
analysis. In order to support longitudinal assessments of health status from young childhood
through adolescence, we adopted the same general
structure as the CHIP-AE, except that children do
not report on their disorders, which are asked only
of parents. The parent version of the questions is
not illustrated. Virtually all the items taken from
the CHIP-AE underwent wording, response format, and/or time-frame changes in order to make
them more developmentally appropriate for
younger children.
General method
Three studies using cognitive interviewing methods were undertaken with a convenience sample of
5±11-year-old children focusing on: (1) the optimal
ways to ask children questions about their health;
(2) the most easily understood response formats;
and (3) children's understanding of health concepts and wording of response options. Parents of
children in daycare or after-school programs were
asked by the daycare providers whether they
62
Table 1 provides a summary of the age, gender,
and race distribution of the sample for each of
the three studies. The samples were recruited to
include an equal number of girls and boys and
whites and African-Americans. The samples had a
predominance of children in the 5±7-year-old
range and families of lower and middle socioeconomic status. E€orts were made to select Ôtypical' children in that age range rather than children
who were especially advanced for their age. Older
children were included in the study, but their
numbers were relatively small because of our primary interest in assessing younger children's ability to report on their health.
The methods used to assess children's comprehension and performance were based on
`think-aloud' methodology [39], especially that
described by Valla et al. [7]. The Ôthink-aloudÕ
approach is viewed as a particularly useful qualitative technique for understanding the cognitive
processes that individuals engage in while attempting to respond to questionnaire items [40].
In the standard think-aloud procedure, participants are asked to verbalize their thoughts while
solving problems or responding to questions [40].
We used this methodology to compare children's
understanding of the items with their scaled response. Children were asked to talk to the interviewer or Ôthink aloudÕ while they were
considering their responses and after they ®nished
responding.
Because of the nature of the Ôthink-aloudÕ responses, qualitative methods of analyses were
employed along with some descriptive statistics in
each of the three studies. In Study 3, age-related
di€erences in understanding and responding to the
health questions were examined using Pearson
correlations and one-way analyses of variances
(ANOVAs).
would be willing to have their child participate.
Parents were asked to sign a consent form that
included several examples of the items to be asked.
Children of consenting parents were subsequently
asked if they would like to participate and if they
would permit audiotape recording of the administration. All children were interested and signed
an assent form after the study had been explained
to them. The study protocol was approved by the
Johns Hopkins institutional review board.
All three studies involved administration of the
items to each child individually, by trained interviewers. Several interviewers administered the
items to children, typically in a large classroom
after the end of the school day. All administrations were tape-recorded. A session with a child
took between 20 and 45 min (median = 30 min),
depending on the protocol being administered and
the age of the child. Children were allowed to take
a break during the session, but rarely did.
A total of 114 children were assessed, with only
three sessions terminated because the child was
unable to comprehend the activity. Thirty-®ve
children aged 5±11 years from two kindergarten
and after-school programs participated in Study 1.
Nineteen children from two after-school daycare
centers were involved in Study 2. Sixty children
were involved in Study 3. Forty two were recruited
from after-school programs as described above.
The other 18 children were recruited from medical clinics at Johns Hopkins Hospital's pediatrics
department in order to involve children who were
familiar with poor health. Nine of these chronically ill children had a cardiac condition, seven
had kidney disease, and two had a chronic gastrointestinal condition. Parents were approached
by clinic sta€ regarding their interest in this
research. Those who agreed to be contacted were
called and came with their child to the study oce.
Table 1. Demographics of each study sample
Study
1
2
3
N
35
19
60
n by year of age
5
6
7
8
9
10
11
Boys n (%)
Non-white n (%)
11
2
7
8
6
12
4
5
16
5
2
9
3
1
6
3
2
8
1
1
2
18 (51)
11 (58)
35 (58)
11 (69)
11 (58)
29 (48)
63
Study 1: Questions, methods, and results
Study 1: How do you ask health questions in a way
that engages children's interest and focuses their
attention?
The speci®c questions were: (1) Can children
translate the intensity of their preferences and
frequency of behavior into a scaled response? (2)
Can a character be developed with whom most
children can easily identify?
Illustrations depicting a character with whom
children could identify (based on preliminary
testing of the illustrations with a convenience
sample) and who illustrated the health concept
were used to anchor each end of the response scale
and aid children's understanding. The illustrations
were drawn by a professional cartoonist with input
and feedback from the investigative team. To
avoid problems with having multiple characters
and test versions, we aimed to develop a Ôuniversal
characterÕ who would be age, gender, and race
neutral. This is in contrast to the approach taken
by Valla, Bergeron and colleagues [7, 33] in their
innovative work, which uses di€erent characters
for each racial group and gender and requires
several di€erent characters and multiple versions
of the instrument.
Study 1 methods
Items and administration
Sixteen simple items representing common tasks or
activities (e.g., eating ice cream, cleaning one's
bedroom) were developed so that we could evaluate children's answers using di€erent response
formats, without concerns about their ability to
understand item content. Each item was presented
twice to each child under one of two orientations
(horizontal or vertical). The order of the presentation of the items was counterbalanced across
children. For each item, children were asked to
mark an ÔXÕ along a blank line VAS to indicate
how they thought or felt about the question. The
response options (Ôa Lot', ÔSomewhere in BetweenÕ,
ÔNot at AllÕ) were read to the child but no response
labels appeared on the VAS (see Figure 1). We also
asked questions designed to assess children's ability to recall the frequency of events occurring over
a period of time (e.g., ÔHow many days have you
eaten ice cream in the past week?Õ). Finally, children were asked: if they thought the illustrated
child character was sort of like them, and why; if
the character was a girl or a boy; and if the character was the same age or younger or older than
they were.
For all items, a `think-aloud' procedure was
used in which the child was asked to verbalize why
he/she was marking a particular response.
Study 1 results
In general, even children as young as age ®ve
seemed to be able to use the VAS. They used
both ends and the middle of the scale, and their
responses on the horizontal and vertical presentations were generally consistent, with signi®cant
disagreement only on two items involving the fre-
``How much do you like ice cream?''
Figure 1. ``If you like ice cream a lot, mark an ``X'' on the line near this child who is eating an ice cream cone. If you do not like ice
cream at all, mark an ``X'' on the line near the child who is not eating ice cream. If you feel somewhere in between, mark an ``X'' along
the middle of the line.'' As an example, the item above was administered using the blank line VAS response format. We also assessed
childrenÕs ability to recall events occurring over a period of time (e.g., ``How many days have you eaten ice cream in the past week?''
``What days were those?'').
64
quency of cleaning up one's bedroom. Of the 35
children tested, all preferred the horizontal presentation, typically explained by, ÔBecause that's
how you read.Õ The 5-year-old children were
apparently able to understand the content of the
questions, but they had diculty in understanding
the concept of a Ôweek.Õ Many were unable to state
how many days were in a week or name the days,
and they frequently confused the events of the past
week with those occurring over a longer (or
shorter) period of time.
We found that children aged seven and under
were very concrete in their identi®cation with the
character. For example, when we asked ÔIs this
child sort of like you?Õ several children responded
ÔNo, because she has only four ®ngers.Õ And, to
ÔDo you think this kid could be your friend?Õ one 5year-old girl replied ÔYes, I could like someone
with only four ®ngers.Õ In subsequent versions of
the questionnaire, the cartoon character had the
appropriate number of digits. The gender of the
character was identi®ed, for the most part, as a girl
by the girls, and a boy, by the boys. They typically
identi®ed the child character as being within 1 year
of their own age. No child indicated that any of
the characters were di€erent from themselves
based on race, facial features, or hair.
Study 2: Questions, methods, and results
Study 2: What response formats are most easily
understood by children?
Four questions were posed: (1) Is a straight line
VAS more easily used and understood by children
than a set of discrete response options presented as
circles? (2) If circles are acceptable, can children
use four labeled circle response options or only
three? (3) Are graduated size circles preferred to
same-size circles? (4) Are two illustrations that
ÔanchorÕ each extreme of the responses preferred to
one illustration?
Study 2 methods
Items and administration
Twenty items representing ®ve of the domains
of the CHIP-AE were chosen. Items that presented
the most concern about children's ability to un-
derstand the question were selected. Each item was
presented twice to each child to test ®ve response
formats, with the order of presentation counterbalanced across children. Each type of response
was anchored with an illustration to the right and
left of the response options. Each response format
was used with eight items: a blank line VAS; a
hatched line VAS with three labeled response areas; three labeled equal-size circle response options; four labeled equal-size circle response
options; four graduated circle response options. A
sixth format was tested in which four graduated
circles were presented with one picture placed
above the circles (see Figure 2 for examples of the
questions and response formats).
For all items, the `think-aloud' procedure was
used in which the child was asked to explain why
he/she was marking a particular response.
At the conclusion of the item presentation,
children were asked which of two response options
was easier to answer and which they liked better.
The direct comparisons given to the children were:
blank line VAS vs. hatched line VAS; VAS vs.
circles; three same-sized circles vs. four same-sized
circles; same-sized circles vs. graduated circles; two
pictures vs. one picture.
Study 2 results
The majority (74%) of children preferred the circle
responses over the VAS lines. Moreover, 68%
preferred the graduated circles over the same-sized
circles, 74% preferred four over three circles, and
58% preferred two rather than one illustration.
Agreement between each child's two responses to
the same items showed that children's responses
were consistent 80% of the time on just over half
the items when the graduated circles and same-sized
circles were presented. On three-fourths of the items
they agreed 80% of the time when four vs. three
circles were presented, whereas only one in four items
had more than 80% agreement in either comparison
in which the VAS format was involved. With only 19
children and four items for each comparison, reliable
statistical estimates are not possible.
Study 3: Questions, methods, and results
Study 3: How well do children understand speci®c
concepts of health? How well do they understand
65
``In the past week, how often did you have a stomachache?''
Figure 2. ''If you had a stomachache every day, mark an ``X'' on the line near this child who looks like (he/she) has a stomachache. If
you had a stomachache no days, mark an ``X'' on the line near this child who looks like (he/she) does not have a stomachache. If you
had a stomachache some days, mark an ``X'' somewhere in between. The closer the ``X'' is to this child [point to child with
stomachache], the more days you had a stomachache. The closer you mark the ``X'' to this child [point to child without a stomachache], the fewer days you had a stomachache.'' ``If you never had a stomachache, mark an ``X'' in the small circle near this child
who does not look like (he/she) has a stomachache. If you sometimes had a stomachache, mark an ``X'' in the circle next to the smaller
circle. If you often had a stomachache, mark an ``X'' in the circle next to the largest circle. And if you always had a stomachache, mark
an ``X'' in the largest circle near the child who looks like (he/she) has a stomachache.'' (Several other items always separated the two
presentations of an item.)
the wording of response formats? How many
response options do children prefer? Is a speci®c
recall period helpful?
Study 3 methods
Items and administration
Thirty two items from the CHIP were presented
twice to test alternative wordings. Two risk
behavior items were asked only of the 8±11year-olds. Four graduated circles, anchored by
illustrations at each end, were used for these items.
Six items were repeated at the end to test children's ability to use ®ve instead of four response
options.
The interviewer presented each item to a child,
directing him/her to mark the circle that was most
true for him/her. After marking each response,
children were asked why they responded the way
they did, and then were asked to explain the
meaning of one key term for each item. Several
examples are, `healthy,' `energy,' `pain,' `threatened,' `shoplifted,' and `on a dare.' The interviewer
presented synonyms and requested examples as
needed to probe children's understanding of the
items. After the six items with ®ve responses were
presented, children were asked: if they thought
they responded the same both times; if it was easier
to answer with four or ®ve circles, and why; and if
they liked four or ®ve circles better, and why.
Scoring and analysis
To examine children's understanding of the key
terms for each item, a three-point coding scheme
66
was developed where 1 = poor or no understanding of the term, 2 = some understanding, and
3 = clear understanding. Children's explanations
of the key terms were coded using the interviewers'
notes, referring back to the tape recordings as
necessary to clarify responses. Inter-rater reliability in coding was 78%. Analysis of the data focused on ®ve areas of interest:
(1) level of understanding of key terms by age and
for the total group;
(2) tendency to select the extreme responses;
(3) tendency to select the more engaging (healthier,
happier) illustration;
(4) preference for a response format of four or ®ve
graduated circles; and
(5) use of a 4-week recall period.
(1) Level of understanding. Analysis of the degree
of understanding of the 24 key terms presented to
children of all ages showed expected age-related
trends; the percentage of terms for which there was
poor understanding varied inversely with age
(Pearson r ˆ 0:70). For the total group 17.4% of
terms were poorly understood. Five-year-olds had
poor understanding of 50.0% of the terms; 6-yearolds understood 25.3% of the terms poorly;
7-year-olds understood 19.0% poorly, and the
older children (ages 8±11) had poor understanding
of only 3.5% of the terms tested. Understanding
by age was signi®cantly di€erent by one-way
analysis of variance (p<0.001, df = 3; F = 27.1).
In post-hoc tests, only the comparison between
ages six and seven was not statistically signi®cant.
Similarly, the percentage of terms that were
clearly understood increased directly with children's age (r=0.69). For the total sample 57.9% of
terms were clearly understood. Five-year-olds
clearly understood 26.8% of terms; 6-year-olds,
47.2%; 7-year-olds, 55.2%, and 8±11-yearold children clearly understood 73.5% of the terms
presented (p<0.001, df = 3; F = 16.2), with no
di€erence between ages ®ve and six, and ages six
and seven). No di€erences were found between the
age groups for the percentage of terms for which
children had ÔsomeÕ understanding. There were no
signi®cant di€erences in understanding between
boys and girls.
Several key terms were identi®ed as problematic
for at least some of the younger children. The word
`healthy' was not understood by a majority of 5year-olds. Younger children and many older children equated `healthy' only with healthy behaviors,
most particularly eating fruits and vegetables.
Table 2 summarizes the percentage of children
at ages 5±11 years with poor understanding of
each of the terms and the rank order of the whole
group's understanding of the terms tested. The age
gradient in understanding is clear, showing that
almost all or all of the older children were able to
understand each of the terms posed to them and
that more than a third of the 5-year-olds did not
understand the majority of these words or phrases.
The 6- and 7-year-olds understood more than the
5-year-olds but they still had trouble with many
concepts. Terms that were not adequately understood by 6- and 7-year-olds included: `threatened,'
`for excitement,' `on a dare,' `shoplifted,' and
`feeling comfortable around others.' The riskbehavior items were asked only to 8±11-year-olds,
all of whom understood the word `weapon,' but
the 8-year-olds did not understand the phrase `to
get high.'
There were no statistically signi®cant di€erences in understanding between the chronically ill
and community samples in Study 3. There were
di€erences in terms of mean responses, indicating
a trend for the chronic illness sample to have
lower satisfaction with health and greater discomfort (especially irritability and restricted activity) than the community children. No
statistical tests were conducted on these small
subsamples.
(2) Selection of extreme responses. To examine
the range of response options, the percentage of
extreme responses (1s and 4s) that each child gave
for the 28 items answered by all ages was computed. Overall, the mean percentage of extreme
responses for the sample was 63.2% (Pearson
correlation with age ˆ 0.62). By age group, the
mean percentage of extreme responses was 87.1%,
78.9%, and 61.4% for children aged 5, 6, 7, respectively, and 50.4% for those ages eight through
11. Children aged ®ve and six gave signi®cantly
higher percentages of extreme responses than
those aged seven or ages 8±11 (p<0.001, df = 3; F
= 14.1). For girls, the mean percentage of extreme
responses was 67.5% and for boys, 59.6%, a non-
67
Table 2. Percentage of children with poor understanding of key terms for total group and by age
Key term
On a dare
Irritable
For excitement
Get away with
Keep you from doing
Threatened
Proud
Temper
Good things
Energy
Healthy enough
Comfortable
Neighborhood
Active games
Real problem
Are taught
Nervous
Other adults
Numbers
Itch
Healthy
Stomachache
Pain
Worried
1
2
Percentage with poor understanding
Age ®ve
(n=7)
Age six
(n=12)
Age seven
(n=16)
Ages 8±11
(n=25)
Total group
(n=60)
Rank1 in total
group
85.7
71.4
85.7
71.4
71.4
71.4
71.4
42.9
57.1
42.9
42.9
57.1
57.1
42.9
57.1
33.3
42.9
42.9
14.3
28.6
42.9
28.6
14.3
28.6
72.7
50.0
45.5
41.7
41.7
41.7
16.7
33.3
25.0
25.0
41.7
18.2
33.3
33.3
0
45.5
16.7
8.3
8.3
16.7
0
0
8.3
0
43.8
43.8
31.3
26.7
18.8
31.3
37.5
18.8
31.3
18.8
18.8
20.0
6.3
6.3
25.0
6.7
12.5
12.5
21.4
6.3
6.3
12.5
6.3
0
8.7
12.0
8.3
4.2
8.3
0
4.0
12.0
0
8.0
0
4.3
4.2
8.0
4.3
0
0
0
0
0
0
0
0
0
40.4
35.0
31.0
25.9
25.4
25.4
23.3
21.7
20.0
18.3
18.3
17.9
16.9
16.7
15.8
14.5
11.7
10.3
8.8
8.3
6.7
6.7
5.1
3.4
1
2
3
4
52
6
7
8
9
102
11
12
13
14
15
16
17
18
19
20
212
22
23
24
Ranking of key terms by % of children with poor understanding (1 = poorly understood by highest % of children).
Tied with next numeric rank in series.
signi®cant di€erence. Although the 6-year-olds
were not confused by a ®ve-point response format
presented in the last six items, they e€ectively
converted it to a three-point format, using only the
middle and both extremes.
(3) Selection of the more engaging illustration.
To explore whether children might choose the
more engaging of the two contrasting illustrations
simply based on identi®cation with the Ôpositive
face,Õ the number of circles marked adjacent to
positive-looking illustrations was computed. Items
without clearly dichotomous facial expressions
were excluded, leaving 24 items for the analysis.
For the total group, the correlation between age
and number of Ôpositive-faceÕ responses was 0.50.
The mean percentage of such responses for the
total sample was 45.8%. By age group, the mean
percentage of Ôpositive facesÕ for ages 5, 6, 7, and
8±11 were 59.6%, 61.7%, 45.0%, and 35.0%,
respectively. The oldest children gave a signi®cantly lower percentage of such responses than
either of the two youngest groups (p<0.001, df =
3; F = 8.6). Boys had, on average, 42.9% Ôpositive-faceÕ responses and girls, 50.0%, a non-signi®cant di€erence. These results are not as skewed
as the endorsement of extreme responses.
(4) Preferred response format. There was no
indication that children had diculty using ®ve
response alternatives to respond to questions. For
questions that children understood well (How
often do you feel really healthy? How often do you
have a stomachache? How many TV shows a day
do you watch?) responses were consistent between
the four- and ®ve-point administration. Fifty six
percent of children thought they answered the ®vepoint response format the same way they answered
the four-circle option. Sixty two percent said that
they thought the ®ve-circle response alternative
68
was easier to answer than the four-circle alternative, and 67% said they liked the ®ve-point alternatives better than the four-circle response
alternative. Their reasons for liking the ®ve-point
response included: ÔIt gives more chance to give my
answer.Õ and ÔBecause I get more choices.Õ
(5) Use of a 4-week recall period. There was an
indication that children understood and accurately
reported health problems within the recall period
of Ôthe past 4 weeksÕ with the expected age-related
di€erences in performance. Most children (~80%)
could tell the interviewer the Ôlast timeÕ a symptom
or behavior had occurred in the past 4 weeks,
although the youngest children were less likely to
be able to do this than the older children. Fewer
than 20% of the children were unable to remember
the symptom or behavior or were unsure of when
it last occurred. Despite recalling symptoms when
asked, it is not clear whether the youngest children
used the time interval appropriately since many
did not understand the concept of a week or a
month.
General issues
All children remained involved in the health survey
task with an interviewer for at least 30 min, many
for 45 min. Children 6 years and older were generally able to quickly understand what they were
supposed to do and that they were to think about
their own health. The 5-year-olds, on the other
hand, often needed extra guidance to understand
what was being asked of them. For illustrations
depicting a speci®c representation of a more general concept (e.g., breaking a rule), young children
were overly focused on the speci®c example provided by the illustration.
Discussion
Signi®cant age-related di€erences in understanding the items and response formats were observed.
Five- to seven-year-old children, especially 5-yearolds, had fundamental problems in understanding
many basic health concepts, dramatically worse
than children aged 8±11. The 5-year-olds needed
much assistance with the tasks, did not understand
the majority of the key terms, and tended to use
only the most extreme responses, e€ectively
describing aspects of their health as good or poor.
Although the 6- and 7-year-olds also had diculty
with some terms, they understood the basic nature
of the health survey and the items and responded
in ways that seemed meaningful, based on their
explanations of why they answered as they did.
Nonetheless, they also tended to use extreme responses. As expected, the terms that presented the
most problems to the younger children were those
that were most abstract, such as `healthy,' `irritable,' and `energy.' The 8±11-year-old children were
almost universally able to understand the nature of
the task and the terms presented to them. They
were comfortable with up to ®ve response options
and explained their answers in ways that clearly
showed they understood what they were saying.
The addition of a speci®c 4-week recall period to
items regarding the experience of symptoms and
behaviors virtually eliminated responses that
referred to distant experiences. Most older children were able to locate the speci®c occurrence of
symptoms or behaviors within the 4-week time
frame. No pattern of gender di€erences in understanding or in use of response options was
found. The majority of the children, regardless of
age, preferred the circles over the VAS, and preferred the graduated circles over the same-size
circles.
Children identi®ed with the illustrated child
character, validating its gender, age, and race
neutrality. They were positively engaged by the
illustrated format of the item presentation and
preferred a response format in which the increasing frequency or intensity of response corresponded with the increasing size of the graduated
circles.
These results suggest that middle elementary
school-aged children can be directly assessed in
terms of their health when asked in a format that
they ®nd acceptable and understandable. Children
as young as age eight were able to report on
virtually all aspects of their health, although the
reliability and validity of these self-reports remains
to be demonstrated. Children aged ®ve had great
diculty understanding task requirements, suggesting the need for further cognitive testing and
use of alternative item and response formats with
children this young. Developmental cognitive
69
limitations were likely responsible for their lack of
comprehension of the task demands [41, 42].
The primary limitation of these cognitive interviews is the small numbers of children assessed in
each phase and at each age. Still, the samples were
adequate to provide guidance in the practical
decisions involved in developing questions to ask
children in this age range. Although the sample
was not representative of a particular group of
children, at least half of the children in each
sample were from families in very low socio-economic circumstances, those youth who were
expected to have the most diculty with these
tasks.
Several areas of understanding are in need of
further study. The children's understanding of
speci®c recall periods was queried many times in the
process of these interviews, indicating that many of
the youngest children could not adequately describe a week or a month. However, a speci®c time
frame was clearly necessary in order to focus children on more recent events. More research is
needed to elucidate how children apply the time
frame posed in a question. Other ways of asking
about timing of events should be investigated such
as `When was the last time you . . .?'
Further research is being conducted in the current ®eld tests of the questions that will ultimately comprise the Child Health and Illness
Pro®le-Child Edition (CHIP-CE). This work will
allow us to more de®nitively address the reliability
and validity of children's self-report health
assessments.
In summary, we have shown that children as
young as age eight are able to report on all aspects
of their health experiences. Children aged 6±7 had
diculty with some health-related terms and
tended to use extreme responses, but they understood the basic task requirements and were able to
report on their health experiences. Children identi®ed with the illustrated child ®gure in a manner
that validated its gender, age, and race neutrality.
The suitability of the cartoons for children in other
countries and cultures awaits future validation.
Children were engaged by the illustrated format of
the item presentation and preferred a response
format in which the increasing frequency of an
experience corresponded with the increased size of
the graduated circles. They can use a ®ve-point
response format allowing the instrument to be
adequately sensitive to di€erences in health. These
results will help guide the future development and
testing of the CHIP-CE.
Acknowledgements
This work was supported by (what is now) the
Agency for Healthcare Research and Quality
(AHRQ) grant HS07045 and AHRQ also supported the development of the CHIP-AE. The
authors would like to thank the daycare and clinic
sta€ and the parents and children for allowing us
to conduct the health interviews. We would also
like to thank Walt Carr for his expert drawing of
the cartoon illustrations used in the cognitive interviewing and Phyllis Friello for her excellent
comments on the manuscript. Pediatricians Sue
Furth, Marlene Miller, and Trish DeRusso assisted us with recruitment of the clinical samples.
Earlier versions of this paper were presented at the
meeting of the American Psychological Association, San Francisco, August 1998 and at the seventh Survey Research Conference, Williamsburg,
VA, September 1999.
References
1. La Greca AM. Issues and perspectives on the child assessment process. In: La Greca AM (ed), Through the eyes of
the child: Obtaining self-reports from children and adolescents, Boston: Allyn and Bacon, 1990: 3±17.
2. Landgraf JM, Abetz LN. Measuring health outcomes in
pediatric populations: Issues in psychometrics and application. In: Spiker B (ed), Quality of Life and Pharmacoeconomics in Clinical Trials. 2nd edn. Philadelphia:
Lippincott-Raven Publishers, 1996: 793±802.
3. Brewster AB. Chronically ill hospitalized children's concepts of their illness. Pediatrics 1982; 69: 355±362.
4. Perrin EC, Gerrity PS. There's a demon in your belly:
Children's understanding of illness. Pediatrics 1981; 67:
841±849.
5. Harter S, Pike R. The pictorial scale of perceived competence and acceptance for young children. Child Dev 1984;
55: 1969±1982.
6. Breton JJ, Bergeron L, Valla JP, et al. Quebec child mental health survey: Prevalence of DSM-III-R mental
health disorders. J Child Psychol Psychiatry 1999; 40:
375±384.
7. Valla JP, Bergeron L, BeÂrube H, Gaudet N, St-Georges M.
A structured pictorial questionnaire to assess DSM-III-Rbased diagnoses in children (6±11 years): Development,
validity, and reliability. J Abnormal Child Psychol 1994; 22:
403±423.
70
8. Stone WL, Lemanek KL. Developmental issues in children's self-reports. In: LaGreca AM (ed), Through the Eyes
of the Child: Observing Self-Reports from Children and
Adolescents, Boston: Allyn and Bacon, 1990: 18±55.
9. Byrne BM. Measuring Self-Concept across the Life Span.
Washington, DC: American Psychological Association,
1996: 52±58.
10. Burbach DJ, Peterson L. Children's concepts of physical
illness: A review and critique of the cognitive-developmental literature. Health Psych 1986; 5: 307±325.
11. Ross DJ, Ross SA. Childhood pain: The school-aged
child's viewpoint. Pain 1984; 20: 179±191.
12. Harbeck C, Peterson L. Elephants dancing in my head: A
developmental approach to children's concepts of speci®c
pains. Child Dev 1992; 63: 138±149.
13. McGrath PA, Seifert CE, Speechley KN, Booth JC, Stitt L,
Gibson MC. A new analogue scale for assessing children's
pain: An initial validation study. Pain 1996; 64: 435±443.
14. Zeltzer LK, LeBaron S, Richie DM, Reed D, School®eld J,
Prihoda TJ. Can children understand and use a rating scale
to quantify somatic symptoms? Assessment of nausea and
vomiting as a model. J Consul Clin Psych 1988; 56: 567±572.
15. Nelson N. Comprehension of spoken language by normal
children as a function of speaking rate, sentence diculty,
and listener age and sex. Child Dev 1976; 47: 299±303.
16. Donaldson M, Wales RJ. On the acquisition of some
relational terms. In: Hayes JR (ed), Cognition and the Development of Language, New York: Wiley, 1968: 235±
268.
17. Natapo€ J. Children's views of health: A developmental
study. Am J Public Health 1978; 68: 995±1000.
18. Natapo€ J. A developmental analysis of children's ideas of
health. Health Educat Quart 1982; 9: 131±141.
19. Hergenrather JR, Rabinowitz M. Age-related di€erences in
the organization of children's knowledge of illness. Dev
Psychol 1991; 227: 952±959.
20. Neuhauser C, Amsterdam B, Hines P, Steward M. Children's concepts of healing: Cognitive development and locus
of control factors. Am J Orthopsych 1978; 48: 335±341.
21. Gale A, Lynn R. A developmental study of attention.
British J Ed Psych 1972; 42: 260±266.
22. Hagen JW, Hale GH. The development of attention in
children. In: Pick AD (ed), Minnesota Symposia on Child
Psychology Minneapolis: University of Minnesota Press,
1973; 7: 117±140.
23. McKay KE, Halperin JM, Schwartz ST, Sharma V.
Developmental analysis of three aspects of information
processing: Sustained attention, selective attention, and
response organization. Dev Neuropsych 1994; 10: 121±
132.
24. Rebok GW, Smith CB, Pascualvaca DM, Mirsky AF,
Anthony BJ, Kellam SG. Developmental changes in
attentional performance in urban children from eight to
thirteen years. Child Neuropsych 1997; 3: 28±46.
25. Walden TA, Field TM. Discrimination of facial expressions
by preschool children. Child Dev 1982; 53: 1312±1319.
26. Wechsler D. Wechsler Intelligence Scale for ChildrenRevised. New York: Psychological Corporation, 1974.
27. Woodcock RW, Mather N. WJ-R tests of cognitive ability
± standard and supplemental batteries: Examiner's manual.
In: Woodcock RW, Johnson MB (eds), Woodcock-Johnson
Psycho-Educational Battery-Revised, Allen, TX:DLM
Teaching Resources, 1990.
28. Ornstein PA. Children's long-term retention of salient
personal experiences. J Traumatic Stress 1995; 8: 581±605.
29. Schwab-Stone M, Fallon T, Briggs M, and Crowther B.
Reliability of diagnostic reporting for children aged
6±11 years: A test±retest study of the diagnostic interview
schedule for children-revise. Am J Psychiatry 1994; 151:
1048±1054.
30. Gathercole SE. The development of memory. J Child Psychol Psychiat 1998; 39: 3±27.
31. Friedman WJ. The development of children's memory for
the time of past events. Child Dev 1991; 62: 139±155.
32. McGrath PA. Versatile pain measures for children. J
Pediatr Som Med 1991; 6: 175.
33. Valla JP, Bergeron L, Bidaut-Russell M, St-Georges M,
Gaudet N. Reliability of the Dominic-R: A young child
mental health questionnaire combining visual and auditory
stimuli. J Child Psychol Psychiatry 1997; 38: 717±724.
34. Shahinfar A, Fox NA, Leavitt LA. Preschool children's
exposure to violence: Relation of behaviour problems to
parent and child reports. Amer J Orthopsych 2000; 70: 115±
125.
35. Raviv A, Raviv A, Shimoni H, Fox N, Leavitt L. Children
self report of exposure to violence and its relation to
emotional distress. J Appl Dev Psych (in press).
36. Riley AW, Green BF, Star®eld B, Forrest CB, Kang M,
Ensminger M. A taxonomy of adolescent health need:
reliability and validity of the adolescent health and illness
pro®les. Med Care 1998; 36: 1228±1236.
37. Star®eld B, Bergner M, Ensminger M, et al. Adolescent
health status measurement: Development of the child
health and illness pro®le. Pediatrics 1993; 91: 430±435.
38. Star®eld B, Riley AW, Green BF, et al. The adolescent
child health and illness pro®le: A population-based measure
of health. Med Care 1995; 33: 553±566.
39. Ericsson KA, Simon HA. Verbal Reports as Data: Protocol
Analyses, rev edn. Cambridge, MA: MIT Press, 1993.
40. Forsyth BH, Lessler JT. Cognitive laboratory methods: A
taxonomy. In: Biemer PP, Groves RM, Lyberg LE,
Mathiowetz NA, Sudman S (eds), Measurement Errors in
Surveys, New York: Wiley, 1991: 393±418.
41. Piaget J. The origins of intelligence in children. New York:
International Universities Press, 1952.
42. Rebok GW. Life-span Cognitive Development. New York:
Holt, Rinehart & Winston, 1987.
Address for correspondence: George Rebok, PhD, Department
of Mental Hygiene, 624 21205 USA
Phone: +1-410-955-8550; Fax: +1-410-955-9088
E-mail: [email protected]