Public Opinion Quarterly, Vol. 70, No. 3, Fall 2006, pp. 354–374
THE ASSOCIATION OF SURVEY SETTING AND
MODE WITH SELF-REPORTED HEALTH RISK
BEHAVIORS AMONG HIGH SCHOOL STUDENTS
NANCY D. BRENER
DANICE K. EATON
LAURA KANN
JO ANNE GRUNBAUM
LORI A. GROSS
TONJA M. KYLE
JAMES G. ROSS
Abstract This study examined whether the prevalence of selfreported health risk behaviors among high school students varied by
survey setting (school versus home) and mode of administration
(paper and pencil versus computer). Students in grades 9 and 11 were
assigned randomly to one of four conditions—school paper-and-pencil
instrument (PAPI), school computer-assisted self-interview (CASI),
home PAPI, and home CASI. During the spring of 2004, 4,506 students completed identically worded questionnaires based on the Youth
Risk Behavior Survey questionnaire. Logistic regression analyses controlling for sex, grade, and race/ethnicity revealed that setting was
associated significantly with the reporting of 30 of the 55 risk behaviors examined, and mode was associated significantly with the reporting of 7 of the 55 behaviors. For all behaviors with a significant
setting main effect, the odds of reporting the behavior were greater
among students who completed questionnaires at school than among
students who completed questionnaires at home. For all behaviors
NANCY D. BRENER, DANICE K. EATON, and LAURA KANN are with the Division of Adolescent and School
Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease
Control and Prevention (CDC), Atlanta, GA. JO ANNE GRUNBAUM is now with the Division of Adult
and Community Health, National Center for Chronic Disease Prevention and Health Promotion,
CDC. LORI A. GROSS, TONJA M. KYLE, and JAMES G. ROSS are with Macro International (ORC Macro),
Calverton, MD. The authors gratefully acknowledge the valuable suggestions of the expert panel
convened for this study: Lara Akinbami, Brett Brown, Kathryn Chandler, Sonia Chessen, James
Colliver, Michael Errecart (deceased), Joe Gfroerer, Gary Giovino, Art Hughes, Ronaldo Iachan,
Meredith Kelsey, Bronwyn Nichols, Patrick O’Malley, Terry Pechacek, Jim Scanlon, Kenneth
Schoendorf, Judy Thorne, and Charles Turner. Address correspondence to Nancy D. Brener; e-mail:
[email protected].
doi:10.1093 / poq / nfl003
Published by Oxford University Press on behalf of the American Association for Public Opinion Research. All rights reserved.
For permissions, please e-mail: [email protected].
Survey Setting and Mode and Reports of Risk Behavior
355
with a significant mode main effect, PAPI mode students had lower
odds of reporting the behavior than CASI mode students. Because
social measurement research assumes that higher prevalence estimates
are more valid than lower estimates, methodological factors shown to
increase estimates, such as setting and mode, should be considered
when planning surveys.
Several national surveys measure health risk behaviors among adolescent
populations (Grunbaum et al. 2004; Johnston et al. 2004; Substance Abuse
and Mental Health Services Administration [SAMHSA] 2004). Although
these surveys sometimes yield similar prevalence estimates of tobacco, alcohol, and other drug use, often the results differ considerably. Such differences
raise important questions among policymakers, researchers, and other users of
the data: How can these differences be explained? Which survey provides
more valid results? Can survey methods be refined to produce more consistent
results?
To answer these questions, the Office of the Assistant Secretary for
Planning and Evaluation within the Department of Health and Human Services commissioned expert papers (Cowan 2001; Fendrich and Johnson
2001; Fowler and Stringfellow 2001; Harrison 2001; Sudman 2001) that
compared the methodologies of three surveys: the Youth Risk Behavior
Survey (YRBS) (Grunbaum et al. 2004); Monitoring the Future (Johnston
et al. 2004); and the National Survey on Drug Use and Health, formerly
known as the National Household Survey on Drug Abuse (SAMHSA
2004). These papers and other articles that have examined differences
among these surveys (Coggeshall and Kingery 2001; Gfroerer, Wright,
and Kopstein 1997) presented similar conclusions. All authors noted that
although methodological factors such as survey setting, question wording,
and data-processing techniques could account for the differences in
results, methodological studies using experimental designs are needed to
understand fully how these factors affect the reported prevalence of health
risk behaviors.
One such study, conducted in 2002, was designed to examine the effect of
question wording and appeals for honesty on the prevalence of self-reported
risk behaviors (Brener et al. 2004a). In that study, high school students were
assigned randomly to complete one of six versions of a paper-and-pencil questionnaire. Each questionnaire version represented a different combination of
honesty appeal and type of question wording. The study found that when population, setting, questionnaire context, mode of administration, and data-editing
protocols were held constant, varying honesty appeals did not have an effect
on the prevalence of health risk behaviors. The study also found that differences in question wording could create statistically significant differences in
some prevalence estimates, but no one type of question wording consistently
produced higher or lower estimates.
356
Brener et al.
Studies examining how survey setting affects prevalence estimates have found
a higher prevalence of risk behaviors among students who completed questionnaires in school than among students who completed questionnaires at home
(Gfroerer, Wright, and Kopstein 1997; Kann et al. 2002; Rootman and Smart
1985). These results, however, were confounded by differences in sample designs,
sample sizes, and response rates. Another study randomly assigned students in a
single sample to complete questionnaires in school or at home; the results did not
show significant differences in prevalence estimates (Needle et al. 1983).
Computer-assisted self-interviewing (CASI) has been used with increasing
frequency in surveys of health risk behaviors. Two studies conducted among
adolescents in their homes found that the reporting of sensitive behaviors was
more likely when using CASI than when using paper-and-pencil instruments
(PAPI) (Turner et al. 1998; Wright, Aquilino, and Supple 1998). In contrast,
two studies comparing CASI and PAPI modes of administration in a school
setting found no significant differences in risk behavior prevalence between
the two modes (Beebe et al. 1998; Hallfors et al. 2000), although higher prevalence was found among students using the PAPI mode than among students
using the CASI mode when students’ computers were situated close together
(Beebe et al. 1998).
These studies on setting and mode demonstrate that, when holding the
mode of administration constant, prevalence of risk behaviors is equal or
higher when questionnaires are administered in schools compared with when
they are administered in students’ homes. The effect of mode, however, appears
to vary by setting. To date, no study has varied systematically both setting and
mode of administration to understand the effects of each. The purpose of this
study is to examine the effects of setting and mode using random assignment
of respondents to four conditions (school PAPI, school CASI, home PAPI,
and home CASI), while holding other methodological factors constant. Based
on the literature, it is proposed that prevalence estimates of health risk behaviors will be higher among students who complete questionnaires at school
than at home and estimates will be lower among students using PAPI than
those using CASI, although this effect might be modified by a setting-bymode interaction. Specifically, it is proposed that prevalence estimates of
health risk behaviors among students who complete questionnaires in school
will not differ significantly by mode, whereas strong mode effects will be
found among students who complete questionnaires at home.
This study also explores factors that might explain differences in the reporting of risk behaviors by setting and mode. For example, substance use prevalence has been shown to vary depending on the degree of respondent privacy
(Aquilino, Wright, and Supple 2000; Gfroerer 1985; Schutz, Chilcoat, and
Anthony 1994), perceived anonymity (Supple, Aquilino, and Wright 1999),
and trust (Wright, Aquilino, and Supple 1998). These variables, as well as comfort and experience with computers, are assessed in this study to understand how
they might modify any setting and mode effects.
Survey Setting and Mode and Reports of Risk Behavior
357
Methods
SAMPLE
Participants were drawn from 64 schools in 8 states. Education agencies that
had successfully completed a YRBS were selected purposively based on their
willingness to participate and with the goal of creating a total sample of geographically, racially, and ethnically diverse students. Contacts at the education
agencies that agreed to participate each identified between 6 and 11 schools
containing grades 9 and 11 for possible participation. Each participating
school then identified two ninth-grade and two eleventh-grade classes from
which to recruit students for the study. All students in selected classes were
eligible to participate. The four selected classes within each school were
assigned randomly to one of four conditions (school PAPI, school CASI,
home PAPI, or home CASI). This study received clearance from the U.S.
Office of Management and Budget and was approved by institutional review
boards at the Centers for Disease Control and Prevention and Macro International (ORC Macro), the study contractor.
Of the 5,920 students enrolled in the selected classes, 83 percent returned
parental permission forms granting them permission to participate in the
study, 4 percent returned forms denying them permission to participate, and
13 percent did not return permission forms despite multiple reminders to students and their parents. Of the 4,935 students who had permission to participate in the study, 4,517 completed questionnaires, resulting in an overall
response rate of 76 percent of all eligible students. This response rate is the
American Association for Public Opinion Research (AAPOR) response rate 1
(AAPOR 2004). In the home setting, among students whose parents had
granted permission, 3 percent refused to participate, and an additional 5 percent
did not participate because data collectors were unable to schedule an
appointment during the field period (February–May 2004) despite repeated
attempts. In the school setting, 12 percent of students whose parents had
granted permission for participation did not complete questionnaires, generally because they were absent on the day of survey administration.
QUESTIONNAIRE
Students in all four conditions completed an 80-item questionnaire that was
labeled and introduced as a “student health survey.” Seventy of the items were
identical to items on the standard 2003 YRBS questionnaire and assessed
demographic characteristics as well as behaviors related to unintentional injuries and violence, tobacco use, alcohol and other drug use, sexual behaviors,
dietary behaviors, and physical activity. Two additional items assessed how
many days the student had missed school for any reason and without permission
during the 30 days preceding the survey. One item measured the student’s
358
Brener et al.
preference for survey mode: “If you had a choice of taking this survey using a
computer or taking this survey using paper and pencil, which would you
choose?”
The remaining seven items were adapted from other studies (National Institute
on Drug Abuse 1991; Tseng et al. 1998; Wright, Aquilino, and Supple 1998)
and assessed factors that might help explain why the reporting of risk behavior
might vary by setting or mode. Three items assessed factors relevant to setting
and mode: perceived anonymity, perceived privacy, and trust. The other four
items assessed factors related to mode: frequency of computer use, use of
computers for surveys or tests, comfort with computers, and computer privacy
(see table 3). Exact question wording for all questions can be found in the
appendix in the online version of the journal.
DATA COLLECTION PROCEDURES
In all four conditions, surveys were administered by trained data collectors.
The data collectors followed standardized protocols for all conditions, including reading aloud scripts that explained the survey procedures.
School. In classes assigned to the PAPI mode, survey administration followed standard procedures used for the national YRBS (Brener et al. 2004b).
Questionnaires were administered in regular classrooms during a single class
period during the school day. Students recorded their responses in a computerscannable questionnaire booklet. In classes assigned to the CASI mode, data
collectors brought in and set up laptop computers in the classroom for students’ use. In about one-fourth of the CASI mode classes, survey administration was conducted in alternate locations, such as multipurpose rooms or
computer labs, because classroom space either was inadequate or not available
at least 30 minutes before data collection was scheduled to take place.
In the CASI mode, the data collectors provided students with instruction
on how to use the computer to complete the survey, and students completed
three practice questions before beginning the actual survey. These practice
questions allowed students to familiarize themselves with how to select
answers, skip questions they chose not to answer, and return to previous
questions.
Home. For the home setting, data collectors scheduled student appointments outside of school hours to administer the questionnaires. On arrival at
each home, the data collector selected a semiprivate, quiet room to administer
the survey, such as the kitchen. Although the student’s home was the preferred
setting for data collection, about 300 (17 percent) students assigned to the
home setting completed the questionnaire away from the home. These alternate sites were quiet public places such as libraries. Whether the survey was
administered in the home or away from the home, the data collector remained
in the room with the student, but not in view of the survey, in case the student
had questions.
Survey Setting and Mode and Reports of Risk Behavior
359
In the home PAPI condition, students recorded their responses on the same
computer-scannable booklets used in the school PAPI condition. In the home
CASI condition, data collectors provided the same instructions for using the
computer that school CASI mode students received, and students completed
the same practice questions before beginning the actual survey.
DATA ANALYSIS
All PAPI questionnaire booklets were scanned using standard YRBS procedures, and all data were edited for inconsistent and out-of-range responses
according to standard YRBS procedures (Brener et al. 2004b). Of the 4,517
completed questionnaires, 3 were deleted because the respondents selected
identical responses for 15 or more questions in a row, and an additional 8 were
deleted because they did not have 20 valid responses remaining after editing.
The final data set therefore contained 4,506 usable questionnaires.
Five of the 60 risk behaviors assessed were excluded from analyses because
the numbers of students who reported engaging in the behaviors were too small
for stable logistic regression models (Peduzzi et al. 1996). For the remaining 55
behaviors, responses were dichotomized so each question measured whether
students engaged in a risk behavior. For 49 of these behaviors, this coding followed standard YRBS convention, with engagement in the behavior being the
response of interest. For six behaviors, however, coding of the referent departed
from standard YRBS procedures, such that engaging in the behavior was the
referent and failure to engage in the behavior was the response of interest. This
recoding simplified interpretation by allowing all behaviors to be coded as risks.
All analyses used SUDAAN to account for the clustering of students within
classrooms. Chi-square analyses examined differences in demographic and
ancillary variables by condition, and results were considered significant at the
p < .05 level. Logistic regression was used to examine simultaneously the
main effects of setting and mode on the reporting of risk behaviors, controlling for sex, race/ethnicity, and age. Age was coded as a five-level categorical
variable (see tables 1 and 2). These categories corresponded to the response
options on the questionnaire, except that “12 years old or younger” and “13
years old” were collapsed with “14 years old” to become “≤14 years old,”
because only seven students indicated they were younger than 14.
For each risk behavior, an additional analysis tested the effect of the interaction of setting and mode by adding to the model the cross-product of setting
and mode. When the interaction reached significance at the p < .05 level, separate models were run stratified by setting. Similarly, for each risk behavior, a
separate set of analyses tested the effects of the interaction of setting with student sex, age, and race/ethnicity. These analyses were conducted by adding
simultaneously to each model the cross-product of setting and each demographic variable. For each interaction that reached significance at the p < .05
level, separate models were run stratified by the demographic variable.
360
Brener et al.
Summary analyses were conducted by creating a count variable for each of
seven risk behavior categories: injury-related behaviors, tobacco use, alcohol
use, drug use, sexual behaviors, weight control behaviors, and physical activity.
Each composite only included behaviors for which it was possible for all
students to have a “yes” response. For example, condom use at last sexual
intercourse was not included in the sexual behavior composite because this
behavior is reported only among students who are currently sexually active.
Students who had missing data on one or more behaviors included in a given
composite were excluded from that composite. Each composite was then used
as a dependent variable in a series of Poisson regression analyses using PROC
LOGLINK in SUDAAN. As in the logistic regression analyses, each model
examined the simultaneous effects of setting and mode while controlling for
sex, age, and race/ethnicity, and an additional set of analyses was conducted
to examine the effect of the setting-by-mode interaction.
Results
Of the 4,506 usable questionnaires in the sample, 1,153 were administered in
the school PAPI condition, 1,144 in the school CASI condition, 1,157 in the
home PAPI condition, and 1,052 in the home CASI condition. Using AAPOR
response rate 1 (AAPOR 2004), response rates within each condition were
similar: 78.2 percent for school PAPI, 74.0 percent for school CASI, 76.7 percent
for home PAPI, and 75.7 percent for home CASI.
The demographic characteristics of the study sample differed from the
national distribution of ninth- and eleventh-grade students (U.S. Bureau of the
Census 2002). In the study sample, female and black non-Hispanic students
were overrepresented, and white non-Hispanic students were underrepresented (table 1). Students in the study sample also tended to be older than
those in the national distribution, which is likely a function of data collection
timing: the national data were collected in the fall, whereas the study was
conducted in the spring.
Student demographic characteristics across the four survey conditions did
not differ significantly by sex or age (table 1). Differences by race/ethnicity
approached significance (p = .053); further analyses revealed this was attributed to differences by condition in the “other” race/ethnicity category. No difference by condition was found in the percentage of students who reported
missing school for any reason on one or more of the 30 days preceding the
survey (χ2 = 1.93, p = .59) and missing school without permission on one or
more of the 30 days preceding the survey (χ2 = 0.23, p = .97).
An analysis of item nonresponse by condition revealed that, although the
level of nonresponse was low overall, the proportion of missing items varied
by condition. Questionnaires completed in the CASI mode had an average of
0.6 percent of items missing in both the home and school settings; those
56.5
43.5
44.5
36.9
12.1
6.6
21.8
24.9
25.4
24.4
3.5
63.8
15.9
16.0
4.4
36.3
14.6
32.7
12.7
3.6
Study Sample (%)
49.5
50.5
National Distribution (%)
24.8
23.2
22.8
24.6
4.6
44.1
35.7
12.5
7.8
56.5
43.5
PAPI (%)
22.1
23.9
27.8
24.1
2.1
42.9
34.9
13.3
9.0
59.5
40.6
CASI (%)
School Setting
21.1
27.6
24.8
22.6
4.0
45.7
37.8
11.5
4.9
54.6
45.4
PAPI (%)
18.8
24.8
26.4
26.7
3.3
45.3
39.3
11.0
4.4
55.4
44.6
CASI (%)
Home Setting
16.06 (0.20)
17.07 (0.053)
3.39 (0.34)
χ2 (p-value)
Demographic Characteristics of Students in Grades 9 and 11 Nationwide and in the Study Sample, by Survey
Sex
Female
Male
Race
White, non-Hispanic
Black, non-Hispanic
Hispanic
Other
Age (years)
≤14
15
16
17
≥18
Characteristic
Table 1.
Condition
362
Brener et al.
completed in the PAPI mode had slightly higher averages (school = 2.1 percent,
home = 1.6 percent).
Table 2 provides the prevalence of each self-reported risk behavior by study
condition and the adjusted odds ratio (AOR) for the association of setting and
mode with each risk behavior. After adjusting for sex, race/ethnicity, age, and
mode, setting was associated significantly with the reporting of 30 of the 55
risk behaviors. For every risk behavior with a significant setting main effect,
the odds of reporting the risk behavior were greater among students in the
school setting than in the home setting. Mode was associated significantly
with the reporting of 7 of the 55 risk behaviors controlling for sex, race/ethnicity, age, and setting. In every model with a significant mode main effect,
students in the PAPI mode had lower odds of reporting the risk behavior than
students in the CASI mode.
The setting-by-mode interaction was statistically significant in 5 of the 55
models tested (table 2, last column). Models stratified by setting showed that
for three of the five behaviors (not going to school because of safety concerns,
current cigarette use, and purchased cigarettes in a store), the odds of reporting the behavior were lower in the PAPI than in the CASI mode in the school
setting, whereas no significant differences in odds by mode in the home setting were identified (data not shown). For one of the five variables (being at
risk for becoming overweight), the odds were higher in the PAPI than in the
CASI mode in the school setting, whereas no significant difference in odds by
mode in the home setting were found. The odds of lifetime illegal steroid use
were greater among students in the PAPI mode than in the CASI mode at
home (AOR = 2.05; 95 percent confidence interval [CI] = 1.06, 3.95), but no
significant difference in odds by mode at school was found (AOR = 0.74; 95%
CI = 0.45, 1.20).
Analyses examining whether setting effects varied by respondent’s sex,
age, and race/ethnicity yielded few significant results. These interactions
could not be assessed for two risk behaviors (tried marijuana before age 13
and current inhalant use) because errors caused by zero values in frequency
table cells occurred during modeling. Of the remaining 53 behaviors, 7
showed a significant setting-by-sex interaction, with Wald Fs ranging from
3.92 (p = .05) for lifetime cigarette use to 8.01 (p = .01) for having driven after
drinking alcohol. Stratified analyses revealed that for all of these behaviors,
which also included having drunk alcohol before age 13, lifetime marijuana
use, current marijuana use, ever had sexual intercourse, and currently sexually
active, male students had greater odds of reporting these behaviors in the
school setting than in the home setting, whereas female students showed no
significant setting effect. Specifically, AORs for male students ranged from
1.33 (95% CI = 1.05, 1.67) for lifetime cigarette use to 2.25 (95% CI = 1.50,
3.36) for having driven after drinking alcohol.
Two of the 53 behaviors showed a significant setting-by-age interaction:
lifetime alcohol use (Wald F = 2.71, p = .03) and current alcohol use (Wald
Rarely or never wore bicycle helmetsa
Rarely or never wore seat belts
Rode with a driver who had been drinking alcoholb
Drove after drinking alcoholb
Carried a weaponb
Carried a gunb
Did not go to school because of safety concernsb
In a physical fightc
Injured in a physical fightc
Dating violencec
Ever forced to have sexual intercourse
Seriously considered attempting suicidec
Made a suicide planc
Attempted suicidec
Lifetime cigarette use
Smoked a whole cigarette before age 13 years
Current cigarette useb
Purchased cigarettes at a store or gas stationb
Tried to quit smokingd
Current smokeless tobacco useb
Current cigar useb
Variable
85.7
11.7
25.5
4.8
13.9
3.1
3.7
34.1
4.0
8.4
5.3
17.7
13.1
9.1
52.7
13.7
13.7
3.0
56.2
4.1
8.9
89.4
12.2
28.6
10.4
13.0
3.9
7.2
34.6
4.1
10.6
8.5
19.1
14.2
10.4
53.9
16.4
17.7
4.9
64.2
4.4
9.7
84.0
7.3
23.7
3.8
8.6
1.2
5.1
28.1
3.0
5.3
3.8
12.8
8.5
5.8
48.9
11.4
16.3
3.9
54.7
2.2
7.9
85.7
8.4
22.2
7.1
10.2
1.9
5.6
30.5
3.9
9.7
6.3
14.6
10.3
8.0
52.2
14.2
14.7
3.3
59.2
3.0
8.7
1.34
1.69*
1.25*
1.50*
1.60*
2.40*
0.98
1.29*
1.19
1.32*
1.36*
1.36*
1.45*
1.41*
1.15
1.23
1.02
1.13
1.16
1.83*
1.18
(0.97, 1.85)
(1.33, 2.15)
(1.05, 1.49)
(1.10, 2.05)
(1.30, 1.98)
(1.56, 3.70)
(0.73, 1.32)
(1.09, 1.53)
(0.86, 1.66)
(1.05, 1.66)
(1.05, 1.76)
(1.14, 1.61)
(1.17, 1.79)
(1.08, 1.83)
(0.97, 1.37)
(0.99, 1.52)
(0.82, 1.28)
(0.78, 1.63)
(0.82, 1.65)
(1.18, 2.83)
(0.92, 1.52)
95% CI
Setting (Home
Is Referent)
School School Home Home
PAPI
CASI PAPI CASI AOR
Condition (%)
0.81
0.90
0.97
0.48*
0.94
0.69
0.64*
0.92
0.82
0.66*
0.61*
0.90
0.86
0.78
0.93
0.78*
0.92
0.83
0.78
0.82
0.88
AOR
(0.59, 1.11)
(0.70, 1.14)
(0.82, 1.15)
(0.35, 0.65)
(0.77, 1.15)
(0.47, 1.01)
(0.47, 0.86)
(0.78, 1.09)
(0.59, 1.14)
(0.52, 0.84)
(0.47, 0.80)
(0.76, 1.06)
(0.70, 1.06)
(0.60, 1.00)
(0.79, 1.11)
(0.63, 0.97)
(0.73, 1.15)
(0.57, 1.22)
(0.55, 1.13)
(0.53, 1.27)
(0.68, 1.12)
95% CI
Mode (CASI
Is Referent)
0.70
0.16
2.37
0.51
1.24
0.20
4.72*
0.36
0.82
2.63
0.03
0.29
0.60
0.99
0.17
0.09
4.08*
5.73*
0.05
0.08
0.01
Setting × Mode
Interaction
(Wald F)
Table 2. Percentage of Students Reporting Engaging in Health Risk Behaviors by Condition and Adjusted Odds Ratios (AOR)
and 95% Confidence Intervals (CI)
(Continued)
Lifetime alcohol use
Drank alcohol before age 13
Current alcohol useb
Episodic heavy drinkingb
Lifetime marijuana use
Tried marijuana before age 13
Current marijuana useb
Lifetime cocaine use
Lifetime inhalant use
Lifetime methamphetamine use
Lifetime ecstasy use
Lifetime illegal steroid use
Ever had sexual intercourse
Had first sexual intercourse before age 13
Had ≥4 sex partners during lifetime
Currently sexually activee
Alcohol or drug use before last sexual intercoursef
Condom use during last sexual intercoursef
Birth control pill use before last sexual intercoursef
Has been pregnant or gotten someone pregnant
Described themselves as overweight
Were trying to lose weight
Variable
Table 2.
72.5
27.2
36.0
16.7
35.7
7.4
17.2
3.1
10.3
3.2
3.2
2.3
47.7
9.4
13.3
32.5
14.2
72.1
9.4
4.5
31.7
48.6
76.5
28.4
42.9
20.7
37.7
8.6
19.1
4.5
12.2
3.8
4.0
3.0
48.3
7.6
13.4
34.6
18.0
70.3
9.7
5.4
30.2
45.8
62.7
20.2
29.5
15.0
32.2
5.5
14.8
2.5
7.8
1.8
2.6
2.6
43.0
6.1
10.9
29.7
12.5
73.3
11.9
3.7
30.4
46.9
68.4
22.7
31.0
14.4
34.0
6.6
15.2
2.1
6.5
1.7
2.7
1.3
45.2
6.4
11.3
31.0
14.9
73.1
17.2
3.6
32.5
44.8
1.55*
1.44*
1.57*
1.39*
1.24*
1.38*
1.29*
1.63*
1.61*
1.97*
1.38
1.33
1.27*
1.54*
1.36*
1.21*
1.28
0.85
0.70
1.39
0.96
1.02
(1.31, 1.83)
(1.23, 1.68)
(1.32, 1.86)
(1.14, 1.69)
(1.04, 1.48)
(1.07, 1.79)
(1.04, 1.61)
(1.03, 2.57)
(1.23, 2.11)
(1.23, 3.16)
(0.95, 2.00)
(0.89, 1.99)
(1.07, 1.52)
(1.17, 2.03)
(1.10, 1.69)
(1.02, 1.45)
(0.89, 1.82)
(0.64, 1.12)
(0.48, 1.01)
(1.00, 1.93)
(0.84, 1.09)
(0.90, 1.17)
95% CI
Setting (Home
Is Referent)
School School Home Home
PAPI
CASI PAPI CASI AOR
Condition (%)
0.81*
0.90
0.84*
0.88
0.94
0.82
0.93
0.87
0.97
0.95
0.90
1.13
0.97
1.07
0.96
0.94
0.77
0.99
0.81
0.93
1.00
1.14
AOR
(0.68, 0.96)
(0.76, 1.05)
(0.71, 0.99)
(0.72, 1.08)
(0.78, 1.11)
(0.64, 1.05)
(0.75, 1.15)
(0.56, 1.34)
(0.76, 1.23)
(0.59, 1.51)
(0.63, 1.30)
(0.77, 1.65)
(0.81, 1.16)
(0.81, 1.41)
(0.77, 1.19)
(0.79, 1.13)
(0.54, 1.09)
(0.75, 1.32)
(0.57, 1.17)
(0.67, 1.29)
(0.88, 1.14)
(1.00, 1.30)
95% CI
Mode (CASI
Is Referent)
0.02
0.58
2.34
3.29
0.07
0.12
0.39
1.37
2.20
0.27
0.26
5.84*
0.01
0.91
0.10
0.34
0.05
0.02
1.46
0.94
2.08
0.22
Setting × Mode
Interaction
(Wald F)
(Continued)
12.3
5.6
3.8
38.9
74.0
50.1
47.3
54.5
20.0
46.7
17.7
15.1
15.4
7.5
5.5
40.2
73.6
51.9
49.1
47.3
19.4
44.8
14.0
14.4
11.0
3.8
2.7
31.8
76.6
47.8
47.7
43.7
14.5
41.3
15.2
16.5
10.3
4.8
3.8
38.5
78.1
52.4
48.5
45.4
17.4
42.3
16.5
17.3
1.37*
1.56*
1.41*
1.19*
0.82
1.01
1.04
1.33
1.24
1.15
0.99
0.86
(1.13, 1.68)
(1.12, 2.18)
(1.01, 1.96)
(1.02, 1.38)
(0.70, 0.96)
(0.87, 1.17)
(0.90, 1.20)
(0.94, 1.88)
(0.89, 1.72)
(0.99, 1.34)
(0.83, 1.18)
(0.72, 1.02)
95% CI
Setting (Home
Is Referent)
School School Home Home
PAPI CASI PAPI CASI AOR
Condition (%)
0.91
0.79
0.69
0.86
0.98
0.91
0.96
1.18
0.99
1.04
1.10
0.98
AOR
(0.75, 1.10)
(0.58, 1.09)
(0.50, 0.96)
(0.74, 1.01)
(0.83, 1.15)
(0.78, 1.06)
(0.83, 1.11)
(0.83, 1.68)
(0.70, 1.38)
(0.90, 1.22)
(0.93, 1.31)
(0.82, 1.17)
95% CI
Mode (CASI
Is Referent)
2.33
0.06
0.01
2.20
0.26
0.37
0.91
0.32
0.89
0.39
5.31*
0.35
Setting × Mode
Interaction
(Wald F)
NOTE.—Models adjusted for sex, race/ethnicity, and age. Race/ethnicity categories included non-Hispanic white, non-Hispanic black, Hispanic, and other. Age
was entered as a five-level categorical variable with the following categories: ≤14 years, 15 years, 16 years, 17 years, and ≥18 years.
a
Among students who had ridden a bicycle during the 12 months preceding the survey.
b
During the 30 days preceding the survey.
c
During the 12 months preceding the survey.
d
Among students who had smoked during the 30 days preceding the survey, those who tried to quit smoking during the 12 months preceding the survey.
e
Sexual intercourse during the three months preceding the survey.
f
Among currently sexually active students.
g
Did not exercise or participate in physical activities that made students sweat and breathe hard for ≥20 minutes on ≥3 of the 7 days preceding the survey.
h
Did not participate in physical activities that did not make students sweat and breathe hard for ≥30 minutes on ≥5 of the 7 days preceding the survey.
i
Among the students enrolled in PE class.
*p < .05.
Fasted to control weightb
Took diet pills to control weightb
Vomited or used laxatives to control weightb
Participated in insufficient vigorous physical activityg
Participated in insufficient moderate physical activityh
Did strengthening exercises fewer than 3 days/week
Watched more than 2 hours/day of TV
Not enrolled in PE class
Not active in PE classi
Did not play on a sports team
At risk for becoming overweight
Overweight
Variable
Table 2.
366
Brener et al.
F = 4.40, p = .002). For the other behaviors, Wald Fs all were greater than p =
.05. Stratified analyses for lifetime and current alcohol use revealed that, for
both behaviors, students aged 16 years and younger had greater odds of
reporting these behaviors in the school setting than in the home setting,
whereas students aged 17 years and older showed no significant setting effect.
For example, for lifetime alcohol use, AORs for students aged ≤14 years, 15
years, and 16 years were 2.00 (95% CI = 1.44, 2.78), 1.77 (95% CI = 1.31,
2.40), and 1.52 (95% CI = 1.08, 2.13), respectively, while AORs for students
aged 17 years and ≥18 years were 1.05 (95% CI = 0.78, 1.43) and 1.25 (95%
CI = 0.59, 2.61), respectively.
Two of the 53 behaviors showed a significant race-by-setting interaction
(Wald F = 2.69, p = .05 for never or rarely wore bicycle helmets; Wald F =
2.77, p = .04 for ever had sexual intercourse). Stratified analyses revealed that
black students had greater odds (AOR = 2.60, 95% CI = 1.55, 4.36) than students in other race/ethnic groups to report not using bicycle helmets in the
school setting than in the home setting, while no significant setting effect was
seen for the other race/ethnic groups. A similar pattern of results was found
for ever having had sexual intercourse, although Hispanic students also
showed significant setting effects for that variable (data not shown).
Summary analyses using behavior category composites revealed that, controlling for sex, age, race/ethnicity, and mode, setting was associated significantly with five of the seven composites (injury-related behaviors, alcohol
use, drug use, sexual behavior, and weight control behaviors). Adjusted incidence density ratios (IDRs) for setting ranged from 1.08 (95% CI = 1.01, 1.15)
for the weight control composite to 1.32 (95% CI = 1.19, 1.47) for the injury
composite. In addition, three of these composites (injury-related behaviors,
alcohol use, and drug use) also showed significant mode effects when controlling for sex, age, race/ethnicity, and mode, with IDRs for mode ranging from
0.84 (95% CI = 0.76, 0.94) for the injury composite to 0.91 (95% CI = 0.85,
0.98) for the alcohol use composite. All composite results that reached significance followed the same pattern as the significant results for the individual
behaviors. That is, the mean number of reported behaviors in each composite
was greater among students who completed questionnaires at school than
among students who completed questionnaires at home, and the mean number
of reported behaviors was lower for PAPI mode students than for CASI mode
students. None of the setting-by-mode interactions for the composites reached
significance.
ANCILLARY VARIABLES
About half of all students (51.3 percent) reported that they preferred to complete the survey on the computer, while 13.2 percent reported preferring paper
and pencil and 35.5 percent said they had no preference. The preferred mode
varied significantly by condition (χ2 = 509.03, p < .001) (table 3). Students in
35.7
20.6
43.6
9.1
67.4
23.5
15.5
63.9
20.6
43.3
60.6
24.4
15.0
8.7
69.1
22.2
14.1
68.7
17.2
38.6
61.9
24.8
13.3
School
PAPI (%)
51.3
13.2
35.5
Overall (%)
Frequency of Ancillary Variable Responses by Setting and Mode
Preferred mode for taking surveya
Computer
Paper and pencil
It would not matter to me
No perceived anonymityb
Yes
No
Not sure
No perceived privacyc
Yes
No
Not sure
Disagree that most people can be trustedd
Frequency of computer use at school, home, or worke
Every day or nearly every day
A few times a week
A few times a month or less
Response
Table 3.
61.1
24.9
14.0
22.7
47.5
29.7
40.1
9.4
68.9
21.7
71.4
4.5
24.2
School
CASI (%)
62.9
25.7
11.3
10.7
79.5
9.8
34.0
8.1
71.1
20.7
26.0
25.4
48.6
Home
PAPI (%)
Condition
63.1
24.1
12.8
7.0
84.9
8.1
36.7
7.9
69.1
23.0
73.1
1.9
25.0
Home
CASI (%)
13.44 (0.004)
5.36 (0.50)
171.11 (<0.001)
4.24 (0.64)
509.03 (<0.001)
χ2 (p-value)
(Continued)
70.5
9.0
19.4
Overall (%)
72.1
12.3
22.0
School
PAPI (%)
67.4
7.7
19.5
School
CASI (%)
75.4
10.1
19.1
Home
PAPI (%)
Condition
66.8
5.7
16.7
Home
CASI (%)
15.50 (0.002)
26.46 (<0.001)
7.44 (0.06)
χ2 (p-value)
b
If you had a choice of taking this survey using a computer or taking this survey using paper and pencil, which would you choose?
Do you believe that the answers you gave in this survey will be linked with your name?
c
While taking this survey, could anyone besides you see your answers?
d
How much do you agree or disagree with the following statement? Most people can be trusted.
e
How often do you use a computer at school, home, or work? Include activities such as being on the Internet, computer games, and e-mail.
f
(Before today), have you ever used a computer to take a survey or test?
g
How much do you agree or disagree with the following statement? Using a computer to take this survey would make (made) me feel nervous.
h
How much do you agree or disagree with the following statement? Using a computer to take this survey would keep (keeps) this survey from being private.
a
Ever used a computer to take a survey or testf
Agree computer surveys make me nervousg
Agree computer surveys prohibit privacyh
Response
Table 3.
Survey Setting and Mode and Reports of Risk Behavior
369
the CASI mode were more likely than students in the PAPI mode to indicate a
preference for the computer, whereas students in the PAPI mode were more
likely than students in the CASI mode to either prefer PAPI or have no preference. When asked if they thought answers on the survey could be linked with
their name (perceived anonymity), 8.7 percent of all students reported “yes,”
and 22.2 percent reported “not sure.” This distribution did not vary significantly by condition (χ2 = 4.24, p = .64) (table 3). The percentage of students
who reported someone could see their answers during the survey (perceived
privacy) differed significantly by condition (χ2 = 171.11, p < .001). Students
in the school setting were more likely to report someone could see their
answers than students in the home setting. The percentage of students who
reported someone could see their answers also was greater in the CASI mode
than in the PAPI mode at school, but not at home. The percentage of students
who disagreed with the statement “most people can be trusted” (trust) varied
significantly by condition (χ2 =13.44, p = .004), with the greater frequency at
school compared with at home.
Overall, 61.9 percent of students used a computer daily or nearly daily,
whereas 13.3 percent of students used a computer a few times a month or less.
This distribution did not vary significantly by condition (χ2 = 5.36; p = .50).
The percentage of students who had ever used a computer to take a survey or
test varied significantly by condition (χ2 = 15.50, p = .002), with the percentage being greater in the PAPI mode (72.1 percent at school and 75.4 percent at
home) than in the CASI mode (67.4 percent at school and 66.8 percent at
home). More students in the PAPI than in the CASI mode agreed that computer surveys make them nervous (χ2 = 26.46, p < .001). The percentage of
students who agreed that computer surveys prohibited privacy did not differ
significantly by condition, although the analysis did approach significance
(χ2 = 7.44, p = .06). The greatest frequency was observed in the school PAPI
condition (22.0 percent), and the lowest frequency was observed in the home
CASI condition (16.7 percent).
For the 30 risk behaviors with a significant setting effect, we tested whether
the association could be explained by adding to the model the three ancillary
variables that might explain why the reporting of risk behaviors varied by setting (perceived anonymity, perceived privacy, and trust). The main effect of
setting became nonsignificant when these three variables were added to models for 5 of the 30 behaviors examined (drove after drinking alcohol, ever
forced to have sexual intercourse, attempted suicide, lifetime cocaine use, and
vigorous physical activity), suggesting the significant association of setting
with these five risk behaviors may be explained by the ancillary variables.
For the seven risk behaviors with a significant mode effect, we tested
whether the association could be explained by adding to the model the seven
ancillary variables that might explain why the reporting of risk behaviors varied by mode (perceived anonymity, perceived privacy, trust, frequency of
computer use, previous use of computers for surveys or tests, comfort with
370
Brener et al.
computers, and computer privacy). The ancillary variables did not explain the
association for any of the behaviors examined.
We conducted parallel analyses for the five composite variables that
showed a significant setting effect and the three composite variables that
showed a significant mode effect. In no case did adding the ancillary variables
to the model explain the association of the composite with setting or mode.
Discussion
More than half of the behaviors examined in this study, and five of seven
behavior composites, showed a significant setting effect. In every case, students who completed questionnaires in school were more likely to report risk
behaviors than were students who completed questionnaires at home. This
finding is consistent with other studies comparing the prevalence of risk
behaviors in school and home settings (Gfroerer, Wright, and Kopstein 1997;
Kann et al. 2002; Rootman and Smart 1985). Mode effects were weaker than
setting effects, with only 13 percent of behaviors examined, and only three of
seven behavior composites, showing a significant mode effect. In every case,
students who completed questionnaires on the computer were more likely to
report the behaviors than were students who completed paper-and-pencil
instruments. Contrary to expectations, only a few behaviors, and none of the
composites, showed a significant setting-by-mode interaction. None of the
significant interactions followed the pattern suggested by the literature, that
students using CASI in the home setting would be more likely to report risk
behaviors than those using PAPI at home (Turner et al. 1998; Wright,
Aquilino, and Supple 1998), while no such effect would be seen in the school
setting (Beebe et al. 1998; Hallfors et al. 2000).
Regarding the types of behaviors most strongly affected by setting and
mode, the results of this study are similar to those of previous studies. Kann
et al. (2002) found setting effects were strongest for illegal or socially stigmatized behaviors, such as drug use and sexual intercourse before age 13,
whereas no significant setting effects were found for less sensitive behaviors,
such as physical activity. Similarly, setting effects in the current study were
less likely to be seen for behaviors related to tobacco use and physical activity
than for behaviors related to violence, suicide, alcohol use, drug use, sexual
behaviors, and unhealthy weight control. In addition, behaviors that showed
mode effects in the current study all were related to injury, alcohol, and drug
use, rather than physical activity, tobacco use, and weight control. These
results are consistent with studies that demonstrate that mode effects are stronger for more sensitive behaviors (Turner et al. 1998; Wright, Aquilino, and
Supple 1998).
Despite what has been suggested in the literature, this study generally found
that perceived anonymity, perceived privacy, trust, and comfort and experience
Survey Setting and Mode and Reports of Risk Behavior
371
with computers did little to explain setting and mode effects. This null result
might be explained by the finding that these ancillary variables did not always
vary by setting and mode in the assumed direction. For example, Sudman
(2001) noted that differences in survey results might best be explained by
greater perceived anonymity in a school-based PAPI survey than in a CASI
survey in which the data collector knows the respondent’s identity, as is true
in the home setting. Supple, Aquilino, and Wright (1999) found adolescents
who completed questionnaires on a computer perceived more response anonymity than adolescents who completed paper-and-pencil questionnaires. The
current study, however, found perceived anonymity did not vary by setting or
mode. Similarly, although Harrison (2001) suggested that lower reported
prevalence of risk behaviors could stem from less perceived privacy in the
home than at school, this study found perceived privacy to be lower among
students who completed surveys at school than at home. Even ancillary variables that varied by setting or mode generally did not modify the effect of setting or mode on the reporting of risk behaviors. Students’ level of trust was
higher in the school setting than the home setting, but that variable did not
modify the association between setting and the reporting of risk behaviors.
This is contrary to previous research, which found mistrust modified mode
effects among young adults (Wright, Aquilino, and Supple 1998). In addition,
although students in the PAPI mode were more likely than students in the
CASI mode to agree that computer surveys made them nervous, this variable
did not modify the association of mode with the reporting of risk behaviors.
One major limitation of this study is that the comparison between the
school and home settings is confounded with group versus individual administration. All students in the school setting completed questionnaires in a group,
whereas all students in the home setting completed questionnaires individually. This study cannot determine, therefore, whether students in the home
setting are less likely to report risk behaviors because being at home increases
the chance that a parent might see their responses or because they are completing the questionnaire individually and the data collector knows their name.
Since perceived anonymity did not vary by setting, the former explanation
may have more validity. This study did not assess whether a parent was
present during survey administration, so it is not possible to determine the
effect of parental presence on the reporting of risk behavior, but parental presence during survey administration has been shown to be associated with lower
reported risk behaviors (Aquilino, Wright, and Supple 2000; Gfroerer 1985;
Schutz, Chilcoat, and Anthony 1994).
This study also cannot determine whether students are more likely to report
risk behaviors in a school setting because they are in the presence of peers or
because of the perceived anonymity that any group provides. For example, asking students about tobacco use in a classroom setting where they are surrounded
by peers might remind respondents of instances in which smoking occurred,
assuming they smoke with their peers (Vilsaint et al. 2003). Others also have
372
Brener et al.
suggested that, to gain increased status among peers, students might overreport
drug use when in the presence of peers (Percy et al. 2005). Thus, the influence
of the school setting on reported risk behaviors likely goes beyond the perceived
anonymity of being in a group. Further research is needed to disentangle the
effect of setting with the effect of group versus individual administration.
This study also is limited by the use of a convenience sample. The demographic characteristics of the study sample differed from those of the national
distribution of ninth- and eleventh-grade students (U.S. Bureau of the Census
2002), although analyses controlled for these characteristics. In addition,
because this study sampled students in grades 9 and 11 only, the ages of the
students not only were restricted to a narrow range but also were not evenly
distributed. These factors limit the study’s ability to detect age differences in
the results. Future research using a broader and more evenly distributed age
range could examine this issue.
Another limitation is the variability of survey logistics within each condition.
Not all students assigned to the home condition completed questionnaires in
their homes, and not all students assigned to the school CASI condition completed questionnaires in a classroom. In the school CASI condition, the location could have affected the distance between students’ computers, which has
been shown to affect the reporting of risk behaviors (Beebe et al. 1998).
Unfortunately, we were not able to collect usable data on these logistics, so we
cannot determine their effects.
Social measurement research assumes respondents underreport sensitive
behaviors, such as health risk behaviors, when data are collected via self-report.
Therefore, higher prevalence estimates are considered more valid (Gans and
Brindis 1995; Moskowitz 2004; Turner et al. 1998). Biochemical measures of
smoking prevalence provide some empirical evidence of this, at least for
tobacco use (Hedges and Jarvis 1998). Consequently, when methodological factors such as setting and mode are shown to increase prevalence estimates of
health risk behaviors, these factors should be considered when planning surveys. This study has shown that prevalence estimates of many health risk behaviors were higher in a school setting than in a home setting. In addition, this study
has shown that setting has a greater effect on risk behavior reporting than does
mode; this finding is consistent with a recent analysis of self-reported tobacco
use (Moskowitz 2004). Use of CASI rather than PAPI in school settings, therefore, might not be justified given the complicated logistics and increased cost.
References
American Association for Public Opinion Research (AAPOR). 2004. Standard Definitions: Final
Dispositions of Case Codes and Outcome Rates for Surveys. 3d ed. Lenexa, KS: AAPOR.
Aquilino, William S., Debra L. Wright, and Andrew J. Supple. 2000. “Response Effects Due to
Bystander Presence in CASI and Paper-and-Pencil Surveys of Drug Use and Alcohol Use.”
Substance Use and Misuse 35:845–67.
Survey Setting and Mode and Reports of Risk Behavior
373
Beebe, Timothy J., Patricia A. Harrison, James A. McCrae, Jr., Ronald E. Anderson, and Jayne A.
Fulkerson. 1998. “An Evaluation of Computer-Assisted Self-Interviews in a School Setting.”
Public Opinion Quarterly 62:623–32.
Brener, Nancy D., Jo Anne Grunbaum, Laura Kann, Tim McManus, and Jim Ross. 2004a.
“Assessing Health Risk Behaviors among Adolescents: The Effect of Question Wording and
Appeals for Honesty.” Journal of Adolescent Health 35:91–100.
Brener, Nancy D., Laura Kann, Steven A. Kinchen, Jo Anne Grunbaum, Laura Whalen, Danice
Eaton, Joseph Hawkins, and James G. Ross. 2004b. “Methodology of the Youth Risk Behavior
Surveillance System.” Morbidity and Mortality Weekly Report 53(RR-12):1–13.
Coggeshall, Mark B., and Paul M. Kingery. 2001. “Cross-Survey Analysis of School Violence
and Disorder.” Psychology in the Schools 38:107–16.
Cowan, Charles D. 2001. “Coverage, Sample Design, and Weighting in Three Federal Surveys.”
Journal of Drug Issues 31:599–614.
Fendrich, Michael, and Timothy P. Johnson. 2001. “Examining Prevalence Differences in Three
National Surveys of Youth: Impact of Consent Procedures, Mode, and Editing Rules.” Journal
of Drug Issues 31:615–42.
Fowler, Floyd Jackson, Jr., and Vickie L. Stringfellow. 2001. “Learning from Experience:
Estimating Teen Use of Alcohol, Cigarettes, and Marijuana from Three Survey Protocols.”
Journal of Drug Issues 31:643–64.
Gans, Janet E., and Claire D. Brindis. 1995. “Choice of Research Setting in Understanding
Adolescent Health Problems.” Journal of Adolescent Health 17:306–13.
Gfroerer, Joseph. 1985. “Influence of Privacy on Self-Reported Drug Use by Youths.” In SelfReport Methods of Estimating Drug Use: Meeting Current Challenges to Validity, ed. Beatrice
A. Rouse, Nicholas J. Kozel, and Louise G. Richards, pp. 22–30, DHHS Publication no.
(ADM) 85–1402. Rockville, MD: National Institute on Drug Abuse.
Gfroerer, Joseph, Doug Wright, and Andrea Kopstein. 1997. “Prevalence of Youth Substance
Use: The Impact of Methodological Differences between Two National Surveys.” Drug and
Alcohol Dependence 47:19–30.
Grunbaum, Jo Anne, Laura Kann, Steve Kinchen, James Ross, Joseph Hawkins, Richard Lowry,
William A. Harris, Tim McManus, David Chyen, and Janet Collins. 2004. “Youth Risk Behavior Surveillance—United States, 2003.” CDC Surveillance Summaries (May 21). Morbidity
and Mortality Weekly Report 53(SS-2):1–96.
Hallfors, Denise, Shereen Khatapoush, Charles Kadushin, Kim Watson, and Leonard Saxe. 2000.
“A Comparison of Paper vs. Computer-Assisted Self Interview for School Alcohol, Tobacco,
and Other Drug Surveys.” Evaluation and Program Planning 23:149–55.
Harrison, Lana D. 2001. “Understanding the Difference in Youth Drug Prevalence Rates
Produced by the MTF, NHSDA, and YRBS Studies.” Journal of Drug Issues 31:665–94.
Hedges, Barry, and Martin Jarvis. “Cigarette Smoking.” In Health Survey for England: The
Health of Young People, 1995–1997, ed. Patricia Prescott-Clarke and Paola Primatesta,
pp. 191–221. London: The Stationery Office.
Johnston, L. D., P. M. O’Malley, J. G. Bachman, and J. E. Schulenberg. 2004. Monitoring the
Future: National Results on Adolescent Drug Use: Overview of Key Findings, 2003. NIH
Publication no. 04–5506. Bethesda, MD: National Institute on Drug Abuse.
Kann, Laura, Nancy D. Brener, Charles W. Warren, Janet L. Collins, and Gary A. Giovino. 2002.
“An Assessment of the Effect of Data Collection Setting on the Prevalence of Health Risk
Behaviors among Adolescents.” Journal of Adolescent Health 31:327–35.
Moskowitz, Joel M. 2004. “Assessment of Cigarette Smoking and Smoking Susceptibility among
Youth: Telephone Computer-Assisted Self-Interviews Versus Computer-Assisted Telephone
Interviews.” Public Opinion Quarterly 68:565–87.
National Institute on Drug Abuse. 1991. National Household Survey on Drug Abuse: Main Findings 1990. DHHS Publication no. (ADM) 91–1788. Rockville, MD: U.S. Department of Health
and Human Services, Public Health Service.
Needle, Richard, Hamilton McCubbin, Jon Lorence, and Mark Hochhauser. 1983. “Reliability
and Validity of Adolescent Self-Reported Drug Use in a Family-Based Study: A Methodological
Report.” International Journal of the Addictions 18:901–12.
Peduzzi, Peter, John Concato, Elizabeth Kemper, Theodore R. Holford, and Alvan R. Feinstein.
1996. “A Simulation Study of the Number of Events per Variable in Logistic Regression Analysis.”
Journal of Clinical Epidemiology 49:1373–79.
374
Brener et al.
Percy, Andrew, Siobhan McAlister, Kathryn Higgins, Patrick McCrystal, and Maeve Thornoton.
2005. “Response Consistency in Young Adolescents’ Drug Use Self-Reports: A Recanting
Rate Analysis.” Addiction 100:189–96.
Rootman, Irving, and Reginald G. Smart. 1985. “A Comparison of Alcohol, Tobacco and Drug
Use as Determined from Household and School Surveys.” Drug and Alcohol Dependence
16:89–94.
Schutz, Christian G., Howard D. Chilcoat, and James C. Anthony. 1994. “Breach of Privacy in
Surveys on Adolescent Drug Use: A Methodological Inquiry.” International Journal of
Methods in Psychiatric Research 4:183–88.
Substance Abuse and Mental Health Services Administration (SAMHSA). 2004. Overview of
Findings from the 2003 National Survey on Drug Use and Health. Office of Applied Studies,
NSDUH series H-24, DHHS Publication no. SMA 043963. Rockville, MD: SAMHSA.
Sudman, Seymour. 2001. “Examining Substance Abuse Data Collection Methodologies.” Journal
of Drug Issues 31:695–716.
Supple, Andrew J., William S. Aquilino, and Debra L. Wright. 1999. “Collecting Sensitive SelfReport Data with Laptop Computers: Impact on the Response Tendencies of Adolescents in a
Home Interview.” Journal of Research on Adolescence 9:467–88.
Tseng, Hsu-Min, Brian Tiplady, Hamish A. Macleod, and Peter Wright. 1998. “Computer Anxiety:
A Comparison of Pen-Based Personal Digital Assistants, Conventional Computer and Paper
Assessment of Mood and Performance.” British Journal of Psychology 89:599–610.
Turner, C. F., L. Ku, S. M. Rogers, L. D. Lindberg, J. H. Pleck, and F. L. Sonenstein. 1998. “Adolescent Sexual Behavior, Drug Use, and Violence: Increased Reporting with Computer Survey
Technology.” Science 280:867–73.
U.S. Bureau of the Census. 2002. “Single Grade of Enrollment and High School Graduation
Status for People 3 Years Old and Over, by Age (Single Years for 3 to 24 Years), Sex, Race,
and Hispanic Origin: October 2002.” Available online at http://www.census.gov/population/
socdemo/school/cps2002/tab02-5.pdf. Accessed July 5, 2006.
Vilsaint, M., M. L. Haviland, J. Allen, J. C. Hersey, K. C. Davis, and P. D. Mowery. 2003. “Do
Peer Context Cues Explain Mode Effects in Youth Surveys?” Paper presented at the annual
meeting of the Society for Research on Nicotine and Tobacco, New Orleans, LA.
Wright, Debra L., William S. Aquilino, and Andrew J. Supple. 1998. “A Comparison of
Computer-Assisted and Paper-and-Pencil Self-Administered Questionnaires in a Survey on
Smoking, Alcohol, and Drug Use.” Public Opinion Quarterly 62:331–53.