Further Validation of the Social and Emotional Health Survey
1
Draft, Submitted for Publication
Further Validation of the Social and Emotional Health Survey for High School Students
SUKKYUNG YOU
College of Education, Hankuk University of Foreign Studies, Seoul, Korea
MICHAEL J. FURLONG AND ERIN DOWDY
Department of Counseling, Clinical, and School Psychology, University of California Santa Barbara, Santa
Barbara, California, USA
TYLER L. RENSHAW
Department of Psychology, Louisiana State University
DOUGLAS C. SMITH
Department of Psychology, Southern Oregon University
MEAGAN D. O’MALLEY
WestEd, Los Alamitos, California USA
Correspondence regarding this article should be addressed to Sukkyung You, Ph.D., College of Education,
Hankuk University of Foreign Studies, 270 Imun-dong, Dongdaemun-Gu, Seoul 130-791, Korea, E-mail:
[email protected]
Further Validation of the Social and Emotional Health Survey
2
Abstract
The Social Emotional Health Survey (SEHS) was developed to assess core building blocks of adolescent positive
psychosocial development and was validated in a previous study (Furlong, You, Renshaw, Smith, & O’Malley,
2013). Using an entirely new sample, the present study extended information about the SEHS convergent and
divergent validity by coadministering it with the Behavioral Emotional Screening System (BESS). The sample
included 2,240 students in Grades 9-12 from two comprehensive high schools located in a major west coast USA
city. A majority of the students were of Latino/a heritage (72%) and were experiencing disadvantaged economic
circumstances (80% at school 1 and 68% at school 2). Predictive validity was examined by comparing SEHS scores
collected at the start of an academic year with the end-of-first-term grade point average. Confirmatory factor
analyses supported the original SEHS factor structure composed of the first-order constructs of belief-in-self, beliefin-others, emotional competence, and engaged living, which parsimoniously mapped on to a second-order
“covitality” factor. Complete factorial invariance was found across four groups formed by crossing gender (male,
female) and age (ages 13-15, ages 16-18). Latent means analysis found some small to moderate effects size
differences, primarily for the belief-in-self and belief-in-others first-order latent traits. An SEM analysis found that
the SEHS measurement model, including covitality was a significant negative predictor of psychological distress as
measured by the BESS and was positively associated with students end-of-term grade point average. The discussion
focuses on implications for conceptualizing the core psychological components of adolescents’ positive quality of
life and how schools can use the SEHS as part of a whole-school procedure to screen for students’ complete mental
health.
Keywords: Social Emotional Health Survey, covitality, belief-in-self, belief-in-others, emotional
competence, engaged living, school, adolescent, complete mental health, validity, confirmatory factor analysis,
latent means analysis, grade point average
Further Validation of the Social and Emotional Health Survey
3
Further Validation of the Social and Emotional Health Survey for High School Students
1. Introduction
1.1 Surveying Students’ Positive Mental Health
School-based surveillance surveys have predominately focused on assessing the nature and frequency of
students’ substance use, aggressive behavior, and other risky behaviors (e.g., Youth Risk Behavior Surveillance
Survey; Eaton et al. 2012). Although these surveys provide useful information that helps schools and their
communities serve high-risk students, their lopsided focus on the negative—or unhealthy—aspects of student
experiences has some limitations. For example, given that many risky behaviors, such as substance use, are either
low prevalence or low frequency behaviors, a majority of the youths are asked to respond to scores of questions that
do not directly match with their personal experiences (e.g., Zhang, Truman, Snyder and Robers 2011). As a result,
students taking these surveillance surveys might begin to question the purpose and relevance of the survey itself,
wondering if it is important to respond carefully (Cornell, Klein, Konold and Huang 2012). Additionally, although
youth involvement in risky behaviors is clearly of public health and educational concern, the predominant
surveillance practices provide limited information about the life conditions and experiences of the majority of youths
who report limited involvement in risky behaviors. This can create circumstances in which school systems are put in
the awkward position of reporting negative results to constituents using a positive spin. For instance, when reporting
the positive information from a surveillance survey that “70% of students did not use marijuana,” concerned parties
such as school board members and parents often state, “So, this means 30% of our students do use marijuana.”
Another limitation of many surveillance surveys is that their scope is unbalanced, focusing mostly on negative
indicators that have been shown to be associated with student distress and school failure, while largely ignoring
indicators of positive youth development that are associated with students’ well-being, school success, and overall
quality of life, even for the youths who are engaging in developmentally risky behaviors (Huebner, Antaramian and
Heffner 2012). If a goal of quality-of-life research is to provoke students, parents, educators, and policy makers to
give greater attention to positive development indicators, then school-based surveillance surveys are needed that
produce conceptually meaningful and practical information about factors that are positively associated with and
promote thriving youth development.
Within the past couple decades, positive youth development as a perspective to enhance youths’ quality of
life has received increasing scholarly attention in several research domains. For example, within the subfield of
youth mental health, studies have found that both the presence of psychological distress and the absence of
psychological well-being are associated with impairments in school performance, and that consideration of both
positive and negative indicators of mental health had additive value in predicting students’ attendance and academic
achievement over time (Antaramian, Huebner, Hills and Valois 2010; Dowdy, Furlong and Sharkey 2013; ShafferHudkins, Suldo, Loker and March 2010; Suldo and Shaffer 2008; Suldo, Thalji and Ferron 2011). Additionally,
results from resilience studies have shown that increased numbers of external assets (e.g., supportive family and
school relationships) and internal assets (e.g., achievement motivation and school engagement) are positively
associated with desirable quality-of-life outcomes, such as academic achievement, physical health, and helping
behaviors, as well as negatively associated with disruptive life outcomes, including substance use, violence
perpetration, and victimization (Evans, Marsh and Weigel 2010; Scales 1999; Scales, Benson, Leffert and Blyth
2000; Scales, Benson, Roehlkepartain, Semsa and Van Dulmen 2006). Moreover, psychological research indicates
that various positive traits (e.g., optimism and school connectedness) are predictive of emotional distress, school
adjustment and success, and physical health (e.g., Boman, Furlong, Shochet, Lilles and Jones 2009; Kirschman,
Johnson, Bender and Roberts 2009). Social-emotional learning (SEL) research has produced empirical evidence
demonstrating that personal abilities or skills (e.g., self-awareness and behavioral regulation) are associated with
better school and quality-of-life outcomes (e.g., Durlak, Weissberg, Dymnicki, Taylor and Schellinger 2011; Zins,
Bloodworth, Weissberg and Walberg 2007). Taken together, these findings across several research domains suggest
that efforts to enhance youths’ quality of life require a balanced approach that includes both efforts to understand
and cultivate positive dimension of youths’ development along with efforts to ameliorate involvement in negative or
unhealthy behaviors.
Given the potential value of considering the combinatorial effects of positive indicators of youth
Further Validation of the Social and Emotional Health Survey
4
development, we have undertaken to validate a psychometrically sound, practically feasible, school-based
assessment as a compliment to risk-oriented surveillance surveys, such as the Youth Risk Behavior Surveillance
Survey (Eaton et al. 2012), and emotional and behavioral problem screening instruments, such as the Behavior
Assessment System for Children–2 Behavioral and Emotional Screening System (BESS; Kamphaus and Reynolds
2007) and the Strengths and Difficulties Questionnaire (Van de Looij-Jansen, Goedhart, de Wilde and Treffers
2011). To accomplish this, in collaboration with WestEd, a nonprofit public research and development agency, and
the California Department of Education we developed and tested the Social and Emotional Health Survey (SEHS)
for secondary students. The SEHS assesses several components of positive mental health—or what we refer to as
positive psychological building blocks—that were drawn from various interrelated subfields of inquiry that are
concerned with positive youth development. Furlong, You, Renshaw, Smith and O’Malley (2013) reported on the
SEHS’s preliminary development and psychometric properties and the present study provides enhanced analyses
using an entirely new sample of adolescents.
1.2 The Social and Emotional Health Survey
The SEHS has 12 subscales, each of which represents a unique positive mental health construct, that
contribute to four positive mental health domains. The first domain, belief-in-self, consists of three subscales
grounded in constructs from the SEL literature: self-efficacy, self-awareness, and persistence (e.g., Durlak et al.
2011; Shechtman, DeBarger, Dornsife, Rosier and Yarnall 2013). The second domain, belief-in-others, is comprised
of three subscales derived from constructs found mostly in the childhood resilience literature: school support, peer
support, and family support (e.g., Larson 2000; Masten, Cutuli, Herbers and Reed 2009). The third domain,
emotional competence, consists of three subscales also based on constructs drawn from the SEL scholarship:
emotional regulation, empathy, and behavioral regulation (e.g., Greenberg et al. 2003; Zins et al. 2007). Engaged
living, the final domain, is comprised of three subscales grounded in constructs derived from the positive youth
psychology literature: gratitude, zest, and optimism (e.g., Gilman, Huebner and Furlong 2009; Kirschman et al.
2009). A representation of SEHS’s subscales and measurement model is shown in Figure 1. A more detailed review
of each of these scales and their associated constructs, and a description of the conceptual rationale underlying the
SEHS, including a discussion of the empirical merit of each of the 12 positive-psychological building blocks, is
provided by Renshaw et al. (in press).
In addition to contributing to the four positive mental health domains described above, the 12 positive
mental health indicators included in the SEHS measurement model are also hypothesized to contribute to a single
latent metaconstruct called covitality. As the counterpart to the comorbidity construct, covitality is conceptualized as
the synergistic effect of positive mental health resulting from the interplay among multiple positive psychological
building blocks. Statistically, covitality is a second-order positive mental health construct that accounts for the
presence of several co-occurring, first-order positive mental health indicators. Findings from our initial study of the
development of the SEHS confirmed the psychometric viability of the 12 positive psychological building blocks, the
four positive mental health domains they contribute to, as well as the overarching covitality construct (Furlong, You,
Renshaw, Smith and O’Malley, 2103). Specifically, results indicated that students’ covitality level was highly
predictive of their subjective well-being (represented by measures of life satisfaction paired with positive and
negative affect) and various self-reported quality-of-life outcomes, including academic achievement, school safety,
depressive symptoms, and substance use (Furlong, You, Renshaw, Smith and O’Malley 2013). These findings are
similar to those reported in other recent studies, where covitality was found to be a parsimonious and effective
predictor of college students’ internalizing symptoms and overall adjustment (Jones, You and Furlong 2012) as well
as elementary-school students’ self-reported prosocial behavior, caring relationships, school acceptance, and school
rejection (Furlong, You, Renshaw, O’Malley and Rebelez 2013).
1.3 Purposes of the Present Study
The purposes of the present study were twofold. First, it aimed to replicate the psychometric characteristics
of the SEHS found in its initial development investigation (i.e., strong factor loadings for the 12 core constructs, the
four positive mental health domains, and the overall covitality construct, as well as complete structural invariance
for both females and males; Furlong, You, Renshaw, Smith and O’Malley 2013). The present study used a new
sample of secondary school students. In the original investigation of the SEHS, the item-response formats for each
Further Validation of the Social and Emotional Health Survey
5
of the 12 subscales were maintained from their respective source measures and, as a result, varied across scales. In
the present study, however, we adopted a more uniform item-response format (described in the Measures section),
reasoning that such standardization might enhance the SEHS’ social validity and interpretability for researchers and
practitioners, and that it might help limit potential invalid response patterns; hence, further clarifying its underlying
conceptual structure. In addition, the present study expanded the original analyses by examining multigroup
invariance across four gender by age groups: (male, female) x (younger adolescents [13-15 years], older adolescents
[16-18 years]). Given Furlong and colleagues’ original findings, we hypothesized that the SEHS latent structure and
overall measurement invariance would be reconfirmed in the present study.
Additionally, this study extended previous research by examining the convergent and divergent validity of
the SEHS with the BESS (Kamphaus and Reynolds 2007), which is a measure of students’ emotional and behavioral
symptoms. Previous research was extended by also examining the relation between students’ covitality level
assessed at the beginning of the school year and actual (instead of self-reported) school grades achieved at the end of
the first school semester. Given Furlong, You, Renshaw, Smith and O’Malley’s (2013) findings regarding the
positive relation of covitality with self-reported academic achievement, we hypothesized that secondary students’
covitality level would predict actual (objective) course grades—suggesting that, in addition to assessing student risk
and problems, schools might find additional benefit from surveying and attending to indicators of students’ positive
mental health factors, as assessed by the SEHS.
2. Method
2.1 Participants
The participants in this study were enrolled in two comprehensive high schools. All students in Grades 9–
12 were invited to participate, with 2,240 of the 2,988 eligible students completing the survey for a participation rate
of 75% (most nonparticipating students were absent on the day of the survey). For the confirmatory factor analyses
described later, two randomly split subsamples (using SPSS Version 20 case selection random sample utility) were
created, with subsample 1 (S1) and subsample 2 (S2) both having 1,120 students. The samples’ gender proportions
were comparable (S1: 46% female, 54% male; S2: 47% female, 53% male). The students ranged in age from 13 to
18 years and both subsamples had the same mean age of 15.5 years (SD = 1.2). A majority of the students (S1:
73%; S2: 71%) reported that they identified as being of Latino/a heritage.
The schools participating in this study were located in a major urban area in Southern California that served
many students who were considered to be economically disadvantaged based on California educational criteria (S1:
80%; S2: 68%). Reflective of these socioeconomic circumstances, these schools had low four-year graduation rates
(S1: 47%; S2: 68%) and inconsistent attendance with low proportions of students having 96% or higher daily school
attendance (S1: 53%; S2: 59%). These schools had stable teaching staffs with a majority of the teachers being at
one of the participating schools for three or more years (S1: 90%; S2: 85%). With respect to campus climate, most
students reported feeling safe at school (S1: 90%; S2: 87%).
2.2 Measures
2.2.1 Social Emotional Health Survey (SEHS)
The SEHS is a modification and extension of the Resilience Youth Development Module (RYDM) that is
part of the California Healthy Kids Survey (see Furlong, Ritchey and O’Brennan 2009; Hanson and Kim 2007).
Furlong, You, Renshaw, Smith and O’Malley (2013) found evidence supporting the multidimensional conceptual
model that motivated its development. The SEHS has 12 subscales (three items per subscale) that map on to the
following four latent traits with the source of the items for each subscale noted: belief-in-self (self-awareness,
Hanson and Kim 1997; persistence, Lufi and Cohen 1987; self-efficacy, Hanson and Kim 2007); belief-in-others
(school support, Hanson and Kim 2007; family coherence, Bloom 1985; peer support, Hanson and Kim 2007);
emotional competence (empathy, Hanson and Kim 2007; self-control, Rohrbeck Azar and Wagner 1991; delay of
gratification, Furlong, Sharkey, Boman and Cladwell 2007); and engaged living (gratitude, Froh, Yurkewicz and
Kashdan 2009; zest, Terry, Lane, Lane and Keohane 1999; optimism, Ey et al. 2005). Support was found for a
structural model that included the single higher-order latent trait, which was called covitality (see Figure 1).
For this study, we slightly modified the response options for 10 of the 12 subscales (excluding gratitude
and zest) to create a more standardized instrument. These response options were: 1 = not at all true of me, 2 = a little
Further Validation of the Social and Emotional Health Survey
6
true of me, 3 = pretty much true of me, and 4 = very much true of me. For the gratitude and zest subscales, the
following response options were used as they referred to frequency of experience: 1 = not at all, 2 = very little, 3 =
somewhat, 4 = quite a lot, 5 = extremely. The 36 SEHS items are shown in Table 1. For the total sample used in the
present study, the internal consistency reliabilities for the four second-order latent traits were: belief-in-self (α =
.76), belief-in-others (α = .81), emotional competence (α = .78), and engaged living (α = .87). The alpha for the
combined 36-item total covitality score was .91, which compares to .92 in a previous study (Furlong, You, Renshaw,
Smith and O’Malley 2013).
2.2.2 Behavioral and Emotional Screening System Student Form (BESS)
The Behavioral and Emotional Screening System Student Form (BESS) is a 30-item self-report measure
designed to identify behavioral and emotional risk in youths Grades 3 through 12. Four response options—never,
sometimes, often, and almost always—are provided for each item and the sum of the items generates a total T score
with higher scores reflecting more problems (Kamphaus and Reynolds 2007). For reviews of the BESS see Furlong
and O’Brennan (2010) and Johnson (2010).
Based on exploratory and confirmatory factor analyses, four subscale scores (including one adaptive scale
and three problem scales) were identified that used 27 of the 30 items in the full BESS Student form (see Dowdy et
al. 2011). Based on responses from the current sample, subscale scores for the three problem scales were calculated
by obtaining the mean of the items that loaded onto each factor. Dowdy et al. (2011) provide a more detailed
description of subscale creation and characteristics and a description of the items that correspond with each subscale.
For the total sample used in the present study, the Cronbach’s alpha for each of the three problem subscales were:
Inattention/Hyperactivity (5 items, α = .61); Internalizing (10 items, α = .80); and School Problems, (4 items, α =
.74).
2.2.3 Course Grading Marks
The SEHS was administered one month after the beginning of the school year’s first semester. We
subsequently obtained first semester course grading marks in January for 1,531 students from one of the
participating high schools. Most of the students (73.9%) took a full schedule of eight courses, with some taking
seven courses (23.0%), six courses (0.4%), and no course marks available for 42 (2.7%) of the students who had
taken the SEHS. Due to the different number of courses the students were enrolled in, we counted the number of A,
B, C, D, and F grades each student had and computed a grade point average (GPA) as follows: ([A Grades * 4] + [B
Grades * 3] + [C Grades * 2] + [D Grades * 1] + [F Grades * 0]) / N of courses. The GPA ranged from 0.0 to 4.0,
with 2.0 equivalent to course grading marks of C’s in all classes.
2.3 Procedure
The SEHS and the BESS were administered in September 2012 to students in a group format during
regular school hours at both high schools on the same day. Survey administration was proctored by university
faculty and trained graduate students. Parental consent was obtained following California educational policy
procedures and as approved by the authors’ institutional review board. Because the survey responses were not
anonymous, the university institutional review board required that a phone call be made to all parents who returned a
signed form indicating that they did not want their child to participate in either the research (n = 8) or the research
and the screening (n = 27). This was to inform the parents that their request was respected. In addition, the students
were fully informed that the survey was not anonymous. They were informed that the survey was being conducted to
assist the student care teams at each school with finding ways to better support students. After receiving the
instructions, the students were informed that they were not required to complete the survey and could opt out with
no repercussions. The SEHS was administered in a paper-and pencil format and students completed the BESS by
using a scanable response form. Class rosters including student names and birthdays were used to provide unique
student identifiers. At the end of the semester, archival data (grades) were collected through an examination of the
student database. Identifying information was removed from all files after data entry, processing, and merging the
grades, SEHS, and BESS responses accomplished using unique student identifiers.
2.4 Overview of the Statistical Analyses
Factor analyses were conducted in two stages. In stage 1, using a split-half of the total sample (S1),
confirmatory factor analysis (CFA) was employed to test the fit of the previously known factor structure of the
Further Validation of the Social and Emotional Health Survey
7
SEHS. In stage 2, using S2, structural equation modeling was used to test two alternative factor structures of the
SEHS and invariance across age by gender groups in a series of multigroup CFAs. The invariance testing process
involved several steps in which increasingly restrictive levels of measurement invariance were explored. Three
levels of measurement invariance were tested in the following order: configural, metric, and scalar (Steenkamp and
Baumgartner 1998). Configural invariance tested if the same basic factor structure held across age by gender
groups. This level of invariance was designed to examine whether the patterns of zero and nonzero factor pattern
coefficients were equivalent across groups and to establish baseline models with adequate fit for the subsequent
measurement invariance testing. This analysis provided information to evaluate if the SEHS subscales fit for the age
by gender groups compared. If configural invariance is established, metric invariance is tested. Metric invariance
tested the extent to which the relations between the factors and the items were equivalent across the groups. This
analysis provided information to evaluate the equivalence of the factor loadings on the SEHS subscales. If latent
factors have equal loadings across groups, this provides evidence that each age by gender group responded to the
items in the same way. Thus, any differences in the latent factors are comparable across groups; that is, observed
group differences in the underlying latent factors. The last step of measurement invariance testing is scalar
invariance. It tested the equality of intercept terms to see whether the four age by gender groups used the response
scales in a similar way. If this invariance test is met, meaningful comparisons of latent means considering age and
gender groups can be made.
In the present study, for all analyses, the degree of model fit was assessed using several criteria: the
Satorra-Bentler scaled statistic, comparative fit index (CFI; Bentler 1990), standardized root-mean-square residual
(SRMR), and root-mean-square error of approximation (RMSEA; Steiger and Lind 1980) with a 90% confidence
interval. Preliminary examination of students’ responses showed that these data were multivariately kurtose; hence,
all analyses were based on robust statistics. Satorra-Bentler scaled statistic (S-B χ2) was used because it provides a
correction to the test statistics and standard errors when data are non-normally distributed. We used the two-index
strategy advanced by Hu and Bentler (1999). Specifically, SRMR was examined with a value lower than .08 desired.
In addition to acceptable SRMR, values lower than .08 for the RMSEA were used to determine a good-fitting model.
In reporting on evidence of invariance, two criterions must be met. The first criterion is that the multigroup model
exhibits an adequate fit to the data. Cheung and Rensvold (2002) recommended that the differences in CFI values
between models are smaller than or equal to -.01, which indicates that the null hypothesis of invariance should not
be rejected. Further, Lagrange multiplier (LM) test modification indices were examined to find which equality
constraints are untenable.
3. Results
3.1 Construct Validity
3.1.1 Confirmatory Factor Analysis
Using S1 (n = 1,120), a CFA was conducted to test the fit of the previously known SEHS factor structure.
The model adequately fit the data, χ2 = 2598.70, df = 582, p < .05; SRMR = .062; RMSEA = .056, 90% CI
[.053, .058]. As expected, each item showed good factor loadings on the corresponding factor. All parameter
estimates were found to be statistically significant (p < .01, see Table 1).
Next, a CFA was assessed on S2 (n = 1,120) using the EQS (V6.1) structural equation modeling program
(Bentler, 2006) to validate the identified factor structure found with the S1 analysis. As with any measure, additional
research is needed to verify the factor structure of the SEHS, therefore, several alternative factor structures were
tested to identify plausible models that could explain the relations among the items. Recognizing that the four core
positive psychology constructs were significantly correlated, Model 1 tested a fully-correlated model. It was found
that the items loaded on to their respective latent constructs: belief-in-self, belief-in-others, social-emotional
competence, and engaged living (all loadings ranged from .41 to .75) and the model adequately fit the data, χ2 =
269.87, df = 48, p < .05; SRMR = .046; RMSEA = .064, 90% CI [.060, .066]. Model 2 extended the analysis by
testing a second-order latent factor model with the four first-order latent constructs loading onto a general latent
factor, which we labeled covitality The model suggested an adequate fit to the data, χ2 = 298.91, df = 50, p< .05,
SRMR = .048; RMSEA = .067, 90% CI [.061, .069]. Both models yielded close fit to the data; therefore, we made a
decision based on both statistical criteria and conceptual grounding. Since parsimony is one of the criteria for model
Further Validation of the Social and Emotional Health Survey
8
selection and the four first-order SEHS constructs loaded significantly on the second-order latent construct,
covitality, the second-order latent factor model was selected over the correlated four-factor model as the preferred
solution. Therefore, the hypothesized second-order factor model was supported for the SEHS and was then used in
the following primary analyses.
3.1.2 Multigroup Invariance Testing
As previously mentioned, measurement invariance testing was performed in three steps: configual, metric,
and scalar. The second-order factor model was conducted using gender by age groups. All four groups showed
adequate fit to the data: age 13-15/male group, χ2 = 97.14, df = 50, p < .05, SRMR = .054; RMSEA = .057, 90% CI
[.052, .060]; age 13-15/female group, χ2 = 113.97, df = 50, p < .05, SRMR = .053; RMSEA = .067, 90% CI [.062,
.070]; age 16-18/male group, χ2 = 110.57, df = 50, p < .05, SRMR = .053; RMSEA = .065, 90% CI [.060, .069]; age
16-18/female group, χ2 = 101.10, df = 50, p < .05, SRMR = .053; RMSEA = .063, 90% CI [.060, .067]. For each of
these four groups, the factor loadings were of all of satisfactory magnitude and were statistically significant at the p
< .01 level. Because the same factor structure was tenable across the four gender by age groups, configural
invariance was ascertained.
Metric and scalar invariance results are displayed in Table 2. First, Model 2 constrained all factor loadings
to be equal across gender groups. Comparing Model 1 (baseline model) and Model 2, ∆CFI was less than .01,
indicating that the latent factors had the same effect on all of their respective observed indicators. Scalar invariance
was evaluated as the last step. The full scalar invariance model held based on acceptable change in ∆CFI. In sum,
the results suggested that the second-order factor model showed sufficient invariance across the four gender by age
groups.
3.1.3 Test of Latent Mean Differences
Given that the assumptions of configural, metric, and scalar invariance were satisfied, the next step tested
for latent means differences. We examined mean differences within the two age groups (ages 13-15 years and 16-18
years) with males set as the reference group for each age group (see Table 3). Results showed there were significant
mean group differences with respect to SEHS factors except for emotional competence; however, the pattern of
significant mean differences differed across the two age groups. For the 13 to 15 years age group, females more
strongly endorsed belief-in-others items than males, but the males more strongly endorsed belief-in-self items than
females. For the age 16 to 18 years group, males reported higher belief-in-self scores than females, but the females
reported higher engaged living than males.
3.2 Predictive Validity
3.2.1 Test of Path Model for Emotional and Behavioral Problems
To further examine the associations among the four identified first-order SEHS constructs, the
hypothesized second-order covitality construct, and the students’ BESS responses, a structural model was conducted
from covitality to the outcome variable—BESS composite problem score (see Figure 2). As expected, the analysis
revealed a significant positive relation to BESS with the overall model having good fit to the data, χ2 = 1085.27, df =
85, p < .05, SRMR = .058; RMSEA = .072, 90% CI [.068, .074]. Figure 2 presents the standardized coefficients of
the final path model.
3.2.2 Course Grade Point Average (GPA)
We first computed a covitality composite score for each student by summing the responses across all 36
items. We found that the total covitality score provided a range of scores both above and below the mean (range =
36–150, M = 112.19, SD = 15.72, skewness = -0.28, kurtosis = -0.04). The covitality scores were then transformed
to z-scores, which were used to form four covitality groups: very low (scores < -.1.0 SD, n = 199), low (scores -1.0 to
0 SD, n = 463), high (score > 0 and < +1.0 SD, n = 531), and very high (scores > +1 SD, n = 255). A one-way
ANOVA tested the relation between students’ covitality level and GPA and yielded a main effect, F (3, 1363) =
18.893, = p < .0001, R2 = 04. Tukey post-hoc tests indicated that the very high and high covitality groups (both with
M = 2.6, SD = 0.8) had higher GPAs than the low (M = 2.3, SD = 0.9) and very low covitality groups (M = 2.2, SD =
0.8).
4. Discussion
Further Validation of the Social and Emotional Health Survey
9
The primary aim of the present study was to replicate and extend analyses that first provided support for the
psychometric properties of the adolescent SEHS. Consistent with a previous study (Furlong, You, Renshaw, Smith
and O’Malley 2013), confirmatory factor analyses replicated the SEHS structure by once again showing that all 36
items loaded uniquely on to their respective 12 base subscales (item loadings ranged from .52 to .90 in the present
study compared with .63 to .94 in a previous study; Furlong, You, Renshaw, Smith and O’Malley 2013), and that the
12 base subscales loaded on to the four proposed first-order latent traits: belief-in-self, belief-in-others, emotional
competence, and engaged living. Given that the SEHS is based on a conceptual model proposing that the
combination of positive psychological building blocks might provide an efficient development index associated with
adolescents’ quality of life, we were encouraged to see that the higher-order structural model including covitality
was supported as the most parsimonious organization of the SEHS latent trait structure.
This study further complemented previous research by showing evidence of divergent validity in that a
structural model including the covitality trait was predictive of students’ self-reported emotional and behavioral
problems, whereas previous research had examined its association with subjective well-being. An additional
contribution was that the present study reported a significant predictive relation with students’ actual course grades.
This provided the first evidence of a link between covitality and an objective, non self-reported indicator, which is
consistent with other studies that have found that students’ subjective well-being (a construct correlated with
covitality) predicts course grades (e.g., Suldo et al. 2011; Suldo, Riley, and Shaffer 2006).
Results of this study provide further evidence of sufficient invariance across four gender by age groups.
These results replicated previous findings of structural invariance for both females and males (Furlong, You,
Renshaw, Smith and O’Malley 2013) using a new sample of adolescents and extended analyses by examining
differences across younger and older adolescents. Although overall measurement invariance was confirmed, some
latent mean differences across gender were found. When compared to males, both younger and older females had
significantly lower self-ratings on the belief-in-self (self-efficacy, self awareness, persistence) items. In contrast,
when compared to males, younger females had higher self-ratings on the belief-in-others (school support, family
coherence, peer support) items and older females had higher ratings on the items assessing engaged living (gratitude,
zest, optimism) These findings are consistent with prior research documenting gender differences in a variety of
these constructs, such as self-efficacy (Meece, Glienke and Burg 2006), social support (Dalgard et al. 2006), and
optimism (Orejudo, Puyuelo, Fernández-Terrado and Ramos 2012). Although the balance of ratings across all
SEHS constructs found no gender differences for younger adolescents and small effect size differences for older
adolescents, additional research is needed to further examine gender and age-related differences in the SEHS.
Conceptual and Research Considerations
Several research domains (positive youth development, resilience, positive psychology, self-determination),
posit that the combination of internal assets have an additive effect as it relates to youth coping and adaptation (e.g.,
Scales et al. 2006). These conceptual perspectives influenced the development of the SEHS, which is an attempt to
provide researchers and educators an efficient measurement that captures the totality of core adolescent
psychological assets. We have proposed that these core psychological assets—belief-in-self, belief-in-others,
emotional competence, and engaged living—represent positive psychological self-schemata (see Dozois et al. 2012;
Markus 1997) that emerge as part of normal youth development and represent the youth’s evolving sense of who
they are and their place within their social niches (Renshaw et al. in press). A possible conceptual implication of
this study’s findings for quality-of-life research is that it reinforces the need for developmental models that are not
built primarily on lower-order, narrow constructs. For example, there is an abundance of meaningful, important
research linking social-emotional development (e.g., social-emotional-learning; Durlak et al. 2011) and positive
traits (e.g., hope [Marques, Lopez, Rose Robinson in press] and gratitude [Bono, Froh Forrett in press]) to positive
mental health, well-being and school success. The SEHS model acknowledges this body of research, and includes
components from its research in its measurement model. In addition, what we are proposing is that it might be
useful to consider how these narrower constructs’ shared variance is linked with higher-order latent traits, as
measured by the SEHS. As an example of this, using a college-age sample Jones et al. (2012) coadministered 40
items from six positive constructs (self-efficacy, hope, life satisfaction, optimism, happiness, and gratitude) and
found that in a cross-instrument factor analysis, 19 items were dropped due to double loadings. By addressing the
Further Validation of the Social and Emotional Health Survey
10
oft overlooked issue of shared measurement error by developing a set of 36 items with high, subscale-unique
loadings, the SEHS offers researchers a possible way to examine correlates and predictors of adolescents’ quality of
life with greater measurement precision. That said, the measurement model underlying the SEHS is not a settled
matter; rather, we offer it as a parsimonious, well-rounded model for capturing core positive-psychological building
blocks that contribute to students’ well-being, school success, and overall quality of life.
Educational Applications
Although the SEHS is grounded in a cognitive organization of adolescent-constructed self-schemata, it can
also be considered within a broader transactional-ecological framework. The SEHS focuses on what adolescents
bring to their various family, school, and community contexts. As youth reach adolescence, it is what they think
about themselves and others that is a particularly vital aspect of their psychological health and quality of life. In
addition to conducting psychometric studies, as a first step to explore the practical applications of the SEHS we have
worked with schools to carry out schoolwide screening for complete mental health (e.g., Dowdy et al. 2013). Using
the SEHS as a complement to more traditional risk surveys, schools have begun to ask students about their
symptoms of mental distress and also about their personal strengths associated with their overall quality of life. For
example, in this study the SEHS and the BESS were coadministered and, when used in combination, offered a more
balanced and comprehensive perspective on student’s complete mental health functioning, with greater attention to
positive development indicators. In addition to the questions being more applicable for all students, the results
provided useful information that informed both whole school and individual-level interventions and supports. Using
the SEHS in conjunction with a risk-focused assessment allows for the assessment of information about constructs
that are related to positive youth development, a practice that is useful and meaningful for schools (see Renshaw et
al. in press).
Study Limitations
Generalization limitations of the current study’s findings need to be considered as it included a sample of
urban Southern California students; however, it did extend support for SEHS structure because the original SEHS
sample included students from a moderate-sized rural community. This study’s sample, and that used in its
preliminary development (Furlong, You, Renshaw, Smith and O’Malley 2013), both had a majority of students who
identified as being of Latino/a heritage. Although there is a need to examine the SEHS factor structure for other
groups of students (both other USA sociocultural groups and cross-national groups), we do note that Latinos now
comprise a majority of California’s high school population and there is building evidence for the SEHS’s validity for
this sizable group of students. Thus far, the samples used in SEHS psychometric studies have included a majority of
students whose families have experienced economic disadvantages. Although it is important that the SEHS is
showing promise as a valid measure of quality-of-life indicators for these higher-risk and higher-need students,
future research is needed that spans across the socioeconomic spectrum. Finally, this study extended the validity of
the SEHS by providing evidence of (a) divergent validity (negative related to BESS scores) and predictive validity
(early school term covitality was positively related to end-of-term course grades). Nonetheless, future research is
needed to examine the relations between the SEHS constructs and other measures of adolescents’ quality of life,
particularly ratings provides by others and actual behaviors.
Conclusion
School personnel are eager to learn about student functioning in order to help students achieve optimal
developmental outcomes. Based on a strong conceptual and empirically-validated model, the SEHS is offered as a
complement to traditional risk-based surveillance systems to provide information on psychological building blocks
that are positively associated with thriving youth development and predictive of academic achievement. There is
now a parsimonious way to measure a variety of constructs associated with positive development, complex models
can be tested efficiently, and results can be applied in a way that is useful for schools. Although additional research
is needed to further understand covitality, school personnel can now confidently use the SEHS to gather valuable
information about student’s social and emotional health.
Further Validation of the Social and Emotional Health Survey
11
References
Antaramian, S. P., Huebner, E. S., Hills, K. J., & Valois, R. F. (2010). A dual‐factor model of mental health: Toward
a more comprehensive understanding of youth functioning. American Journal of Orthopsychiatry, 80, 462–
472. doi:10.1111/j.1939-0025.2010.01049.x
Bentler, P. M. (1990). Comparative fit indices in structural models. Psychological Bulletin, 107, 238–146.
doi:10.1037/0033-2909.107.2.238
Bentler, P. M. (2006). EQS 6 Structural Equations Program manual. Encino, CA: Multivariate Software.
Bloom, B. L. (1985). A factor analysis of self-report measures of family functioning. Family Process, 24, 225–239.
doi:10.1111/j.1545-5300.1985.00225.x
Boman, P., Furlong, M. J., Shochet, I., Lilles, E., & Jones, C. (2009). Optimism and the school context. In R.
Gilman, E. S. Huebner, & M. J. Furlong (Eds.), Handbook of positive psychology in schools (pp. 51–64).
New York, NY: Routledge.
Bono, G., Froh, J. J., & Forrett, R. (in press). Gratitude in schools: Benefits to students and schools. In M. J.
Furlong, R. Gilman, & E. S. Huebner (Eds.), The handbook of positive psychology in the school (2nd ed.).
New York, NY: Routledge/Taylor & Francis.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance.
Structural Equation Modeling, 9, 233–255. doi:10.1207/S15328007SEM0902_5
Cornell, D., Klein, J., Konold, T., & Huang, F. (2012). Effects of validity screening items on adolescent survey data.
Psychological Assessment, 24, 21–35. doi:10.1037/a0024824
Dalgard, O. S., Dowrick, C., Lehtinen, V., Vazquez-Barquero, J., Casey, P., Wilkinson, G. et al. (2006). Negative
life events, social support and gender difference in depression: A multinational community survey with
data from the ODIN study. Social Psychiatry and Psychiatric Epidemiology, 41, 444–451.
doi:10.1007/s00127-006-0051-5
Dowdy, E., Furlong, M. J., Kaufman, B., Raines, T. C., Price, M., Murdock, J. et al. (2013). Enhancing schoolbased mental health services with a preventive and promotive approach to universal screening for
complete mental health. Manuscript submitted for publication.
Dowdy, E., Furlong, M. J., & Sharkey, J. D. (2013). Using surveillance of mental health to increase understanding
of youth involvement in high-risk behaviors: A value added analysis. Journal of Emotional and Behavioral
Disorders, 21, 33–44. doi:10.1177/1063426611416817//
Dowdy, E., Twyford, J. M., Chin, J. K., DiStefano, C. A., Kamphaus, R. W., & Mays, K. L. (2011). Factor structure
of the BASC–2 Behavioral and Emotional Screening System Student Form. Psychological Assessment, 23,
379–387. doi:10.1037/a0021843
Dozois, D. J. A., Eichstedt, J. A., Collins, K. A., Phoenix, E., & Harris, K. (2012). Core beliefs, self-perception, and
cognitive organization in depressed adolescents. International Journal of Cognitive Therapy, 5, 99–112.
doi:10.1521/ijct.2012.5.1.99
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of
enhancing students’ social and emotional learning: A meta-analysis of school-based interventions. Child
Development, 82, 405–432. doi:10.1111/j.1467-8624.2010.01564.x
Eaton, D. A., Kann, L., Kinchen, S., Shanklin, S., Flint, K. H., Hawkins, J. et al. (2012). Youth risk behavior
surveillance—United States, 2011. MMWR, Morbidity and Mortality Weekly Report, 61 (4).
Evans, W. P., Marsh, S. C., & Weigel, D. J. (2010). Promoting adolescent sense of coherence: Testing models of
risk, protection, and resiliency. Journal of Community & Applied Social Psychology, 20, 30–43.
doi:10.1002/casp.1002
Ey, S., Hadley, W., Allen, D. N., Palmer, S., Klosky, J., Deptula, D. et al. (2005). A new measure of children’s
optimism and pessimism: The Youth Life Orientation Test. Journal of Child Psychology and Psychiatry,
46, 548–558. doi:10.1111/j.1469-7610.2004.00372.x
Froh, J. J., Yurkewicz, C., & Kashdan, T. B. (2009). Gratitude and subjective well-being in early adolescence:
Examining gender differences. Journal of Adolescence, 32, 633–650.
doi:10.1016/j.adolescence.2008.06.006
Further Validation of the Social and Emotional Health Survey
12
Furlong, M. J., & O’Brennan, L. (2010). Review of the Behavioral and Emotional Screening System. In R. A. Spies,
K. F. Geisinger, & J. F. Carlson (Eds.), The 18th Mental Measurements Yearbook. Lincoln, NE: Buros
Institute of Mental Measurements.
Furlong, M. J., Ritchey, K. M., & O’Brennan, L. M. (2009). Developing norms for the California Resilience Youth
Development Module: Internal assets and school resources subscales. California School Psychologist, 14,
35–46.
Furlong, M. J., Sharkey, J. D., Boman, P., & Caldwell, R. (2007). Cross-validation of the Behavioral and Emotional
Rating Scale-2 Youth Version: An Exploration of strength-based latent traits. Journal of Child and Family
Studies, 16, 696–711. doi:10.1007/s10826-006-9117-y
Furlong, M. J., You, S., Renshaw, T. L., O’Malley, M. D., & Rebelez, J. (2013). Preliminary development of the
Positive Experiences at School Scale for elementary school children. Child Indicators Research. Advanced
online publication. doi:10.1007/s12187-013-9193-7
Furlong, M. J., You, S., Renshaw, T. L., Smith, D. C., & O’Malley, M. D. (2013). Preliminary development and
validation of the Social and Emotional Health Survey for secondary students. Social Indicators Research.
Advanced online publication. doi:10.1007/s11205-013-0373-0
Gilman, R., Huebner, E. S., & Furlong, M. J. (Eds.). (2009). Handbook of positive psychology in schools. New
York, NY: Routledge.
Greenberg, M. T., Weissberg, R. P., O’Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., & Elias, M. J. (2003).
Enhancing school-based prevention and youth development through coordinated social, emotional, and
academic learning. American Psychologist, 58, 466–474. doi:10.1037/0003-066X.58.6-7.466
Hanson, T. L., & Kim, J. O. (2007). Measuring resilience and youth development: the psychometric properties of
the Healthy Kids Survey. (Issues & Answers Report, REL 2007–No. 034). Washington, DC: U.S.
Department of Education, Institute of Education Sciences, National Center for Education Evaluation and
Regional Assistance, Regional Educational Laboratory West. Retrieved from http://ies.ed.gov/ncee/edlabs
Hu, L., & Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria
versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118
Huebner, E. S., Antaramian, S. P., & Heffner, A. L. (2012). Perceived quality of life research on children and youth:
Implications for a system of national indicators. In K. Land (Ed.), The well-being of America’s children
(Vol. 6, pp. 121–141). Heidelberg, Netherlands. Springer. doi:10.1007/978-94-007-4092-1_5
Johnson, K. M. (2010). Review of the BASC-2 Behavioral and Emotional Screening System. In R. A. Spies, K. F.
Geisinger, & J. F. Carlson (Eds.), The 18th Mental Measurements Yearbook. Lincoln, NE: Buros Institute
of Mental Measurements.
Jones, C. N., You, S., & Furlong, M. J. (2013). A preliminary examination of covitality as integrated well-being in
college students. Social Indicators Research, 111, 511–526. doi:10.1007/s11205-012-0017-9
Kamphaus, R. W., & Reynolds, C. R. (2007). BASC-2 Behavioral and Emotional Screening System manual.
Bloomington, MN: Pearson.
Kirschman, K. J. B., Johnson, R. J., Bender, J. A., & Roberts, M. C. (2009). Positive psychology for children and
adolescents: Development, prevention, and promotion. In S. J. Lopez & C. R. Snyder (Eds.), The Oxford
handbook of positive psychology (2nd ed., pp. 133–148). New York, NY: Oxford.
Larson, R. W. (2000). Toward a psychology of positive youth development. American Psychologist, 55, 170–183.
doi:10.1037/0003-066X.55.1.170
Lufi, D., & Cohen, A. (1987). A scale for measuring persistence in children. Journal of Personality Assessment, 51,
178–185. doi:10.1207/s15327752jpa5102_2
Markus, H. (1977). Self-schemata and processing information about the self. Journal of Personality Social
Psychology, 35, 63–78.
Marques, S. C., Lopez, S. J., Rose, S., & Robinson, C. (in press). Measuring hope in school children. In M. J.
Furlong, R. Gilman, & E. S. Huebner (Eds.), The handbook of positive psychology in the school (2nd ed.).
New York, NY: Routledge/Taylor & Francis.
Meece, J. L., Glienke, B. B., & Burg, S. (2006). Gender and motivation. Journal of School Psychology, 44, 351–
Further Validation of the Social and Emotional Health Survey
13
373. doi:10.1016/j.jsp.2006.04.004
Orejudo, S., Puyuelo, M., Fernández-Turrado, T., & Ramos, T. (2012). Optimism in adolescence: A cross-sectional
study of the influence of family and peer group variables on junior high school students. Personality and
Individual Differences, 52, 812–817. doi:10.1016/j.paid.2012.01.012
Renshaw, A. S., Cutuli, J. J., Herbers, J. E., & Reed, M. G. (2009). Resilience in development. In S. J. Lopez & C.
R. Snyder (Eds.), The Oxford handbook of positive psychology (2nd ed., pp. 117–132). New York, NY:
Oxford.
Renshaw, T. L., Furlong, M. J., Dowdy, E., Rebelez, J., Smith, D. C., & O’Malley, M. D. et al. (in press).
Covitality: A synergistic conceptualization of adolescents’ mental health. In M. J. Furlong, R. Gilman, & E.
S. Huebner (Eds.), Handbook of positive psychology in the schools (2nd ed.). New York, NY: Routledge.
Rohrbeck, C. A., Azar, S. T., Wagner, P. E., (1991). Child Self-Control Rating Scale: Validation of a child selfreport measure. Journal of Clinical Child & Adolescent Psychology, 20, 179–183.
doi:10.1207/s15374424jccp2002_9
Scales, P. C. (1999). Reducing risks and building developmental assets: Essential actions for promoting adolescent
health. Journal of School Health, 69, 113–119. doi:10.1111/j.1746-1561.1999.tb07219.x
Scales, P. C., Benson, P. L., Leffert, N., & Blyth, D. A. (2000). Applied Developmental Science, 4, 27–46.
doi:10.1207/S1532480XADS0401_3
Scales, P. C., Benson, P. L., Roehlkepartain, E. C., Semsa, A., & Van Dulmen, M. (2006). The role of
developmental assets in predicting academic achievement: A longitudinal study. Journal of Adolescence,
29, 691–708. doi:10.1016/j.adolescence.2005.09.001
Shaffer-Hudkins, E., Suldo, S., Loker, T., & March, A. (2010). How adolescents’ mental health predicts their
physical health: Unique contributions of indicators of subjective well-being and psychopathology. Applied
Research in Quality of Life, 5, 203–217. doi:10.1007/s11482-010-9105-7
Shechtman, N., DeBarger, A. H., Dornsife, C., Rosier, S., & Yarnall. L. (2013). Promoting grit, tenacity, and
perseverance: Critical factors for success in the 21st Century. Washington, DC: U.S. Department of
Education, Office of Educational Technology, Center for Technology in Learning, SRI International.
http://www.ed.gov/edblogs/technology/files/2013/02/OET-Draft-Grit-Report-2-17-13.pdf
Suldo, S. M., & Shaffer, E. J. (2008). Looking beyond psychopathology: The dual-factor model of mental health in
youth. School Psychology Review, 37, 52–68.
Suldo, S. M., Thalji, A., & Ferron, J. (2011). Longitudinal academic outcomes predicted by early adolescents’
subjective well-being, psychopathology, and mental health status yielded from a dual-factor model. The
Journal of Positive Psychology, 6, 17–30. doi:10.1080/17439760.2010.536774
Terry, P. C., Lane, A. M., Lane, H. J., & Keohane, L. (1999). Development and validation of a mood measure for
adolescents. Journal of Sports Sciences, 17, 861–872. doi:10.1080/026404199365425
Steenkamp, J-B. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer
research. Journal of Consumer Research, 25, 78–90. http://www.jstor.org/stable/10.1086/209528
Steiger, J. H., & Lind, A. (1980). Statistically based tests for the number of common factors. Paper presented at the
annual meeting of the Psychometric Society, Iowa City, IA.
Van de Looij-Jansen, P. M., Goedhart, A. W., de Wilde, E. J., & Treffers, P. D. A. (2011). Confirmatory factor
analysis and factorial invariance analysis of the adolescent self-report Strengths and Difficulties
Questionnaire: How important are method effects and minor factors? British Journal of Clinical Psychology,
50, 127–144. doi:10.1348/014466510X498174
Zhang, J., Truman, J., Snyder, T. D., Robers, S., & American Institutes for Research. (2012). Indicators of school
crime and safety, 2011. Washington, DC: Bureau of Justice Statistics, Institute of Education Sciences.
https://www.ncjrs.gov/App/Publications/abstract.aspx?ID=258009
Zins, J. E., Bloodworth, M. R., Weissberg, R. P., & Walberg, H. J. (2007). The scientific base linking social and
emotional learning to school success. Journal of Educational and Psychological Consultation, 17, 191–
210. doi:10.1080/10474410701413145
Further Validation of the Social and Emotional Health Survey
14
Table 1
Standardized Factor Loadings for the Covitality Scales from Random Sample 1 (S1).
Items and Scales
Belief-in-Self
Self-Efficacy a
I can work out my problems.
I can do most things if I try.
There are many things that I do well.
Self-Awareness a
There is a purpose to my life.
I understand my moods and feelings.
I understand why I do what I do.
Persistence a
When I do not understand something, I ask the teacher again and again until I understand.
I try to answer all the questions asked in class.
When I try to solve a math problem, I will not stop until I find a final solution.
Belief-in-Others
School Support a (At my school there is a teacher or some other adult who)
…always wants me to do my best.
…listens to me when I have something to say.
…believes that I will be a success.
Family Coherence a
My family members really help and support one another.
There is a feeling of togetherness in my family.
My family really gets along well with each other.
Peer Support a (I have a friend my age who)
…really cares about me.
…talks with me about my problems.
…helps me when I’m having a hard time.
Emotional Competence
Emotional Regulation a
I accept responsibility for my actions.
When I make a mistake I admit it.
I can deal with being told no.
Empathy a
I feel bad when someone gets their feelings hurt.
I try to understand what other people go through.
I try to understand how other people feel and think.
Behavioral Self-Control a
I can wait for what I want.
I don’t bother others when they are busy.
I think before I act.
Engaged Living
Gratitude b
Grateful
Thankful
Appreciative
Loadings
.57
.63
.62
.55
.69
.71
.66
.60
.61
.78
.72
.84
.78
.90
.81
.77
.90
.90
.70
.71
.54
.56
.86
.81
.52
.58
.58
.84
.85
.72
Further Validation of the Social and Emotional Health Survey
15
Zest b
Energetic
.85
Active
.88
Lively
.73
Optimism a
Each day I look forward to having a lot of fun.
.74
I usually expect to have a good day.
.82
Overall, I expect more good things to happen to me than bad things.
.69
__________________________________________________________________________________________
Note. Response prompt for all items: Please mark the response that shows how true each of these statements
is about you.
All values were statistically significant at p < .05. n = 1,120.
a
Response options: 1 = not at all true of me, 2 = a little true of me, 3 = pretty much true of me, 4 = very much
true of me
b
Response options: 1 = not at all, 2 = very little, 3 = somewhat, 4 = quite a lot, 5= extremely
Table 2
Model Fit Indices For Invariance Tests For The Second-Order Factor Model
Model and invariance level
Model 1 (baseline model): Configural invariance
S-B χ2
422.44
df
200
SRMR
.054
*RMSEA (CI)
.063 [.055, .071]
∆CFI
—
Model 2: Full metric invariance
469.47
233
.073
.060 [.052, .068]
.005
Model 3: Full metric and full scalar invariance
805.86
261
.072
.087 [.080, .093]
.008
Note. S-B χ2 = Satorra-Bentler scaled chi-square statistic; SRMR= standardized root-mean-square residual.
*RMSEA = robust root-mean-square error of approximation. CI = 90% confidence interval. ∆CFI = difference in
robust comparative fit indices between baseline model.
Table 3
Results of Structured Means Analyses with Male Group as the Reference Group
Variable
Age 13-15 years
Age 16-18 years
Factor
Effect
Factor
intercept (SE)
Size (d)
intercept (SE)
Belief-in-Self
-.090 (.035)
-.26*
-.171 (.038)
Belief-in-Others
.151 (.046)
.38*
-.010 (.063)
Emotional Competence
.014 (.050)
.03
.011 (.044)
Engaged Living
-.084 (.058)
-.14
.260 (.065)
CoVitality (highest-order latent trait)
.015 (.035)
.06
.086 (.001)
Note. The latent mean values for male group were set to zero. *p < .05.
Effect
Size (d)
-.50*
-.02
.03
.39*
.24*
Further Validation of the Social and Emotional Health Survey
16
Figure 1. Social and
Emotional Health
Survey conceptual
and measurement
model.
Further Validation of the Social and Emotional Health Survey
17
Figure 2. Social and Emotional Health Survey hierarchial CoVitality (CoVi) model (N = 2,240). BESS = Behavioral
Emotional Screening System.