Career and College Success:
Measuring Middle School Student Academic Mindsets and Behaviors
By. Eian Harm, Lisa Austin, and Kevin Thienes
June 2015
Purpose:
This research is an attempt to identify areas of the Career and College Readiness (C&CR) framework
which have improved through the targeted implementation of these projects. In addition, this research
seeks to identify strengths and areas of further work which can enable directed action by school staff
through data-based decisions.
Background:
West Ada School District has a continuing emphasis on developing career and college ready students
who will be successful in any post-secondary endeavor. As a result of this focus, a group of schools
ranging from K-12 have begun to embrace a research-based framework based on a model developed by
Conley (2011). This model uses four (4) factors most highly related to student success through
consecutive levels of education leading up to entrance into a career. These factors are seen in figure 1
below.
Figure 1. Conley’s four “keys” to College and Career Readiness
It is hypothesized that a nearly equal focus in time, effort, and resources should be applied to all areas of
the framework (as needed based on school cultural characteristics) in order to empower students with
the skills necessary to be deemed career and college ready. This particular study is the continuation of
research which began during the 2013/2014 school year within a middle school within the district
(Harm, Austin, & Thienes, 2014). The school’s initial and current focus was on increasing the “Key
Learning Skills and Techniques” category of the framework through working on student growth mindset,
grit, and perseverance as demonstrated by such researchers as Carol Dweck (2006), Stanford PERTS lab
(2015), and the University of Chicago (Farington et al., 2012). In addition to their work on developing
these academic mindsets and behaviors (such as growth mindsets) this school has also begun a project
concentrating on students identifying their personality traits and researching and presenting possible
careers best aligned to these traits. This is in an attempt to align school-wide endeavors with career and
college readiness as suggested by the research (Conley, 2011). These career-based projects were
accomplished by all 8th grade students at the school and provide an effective transition to the content
areas and career choices provided within the high school level.
Methods:
In order to measure student Key Learning Skills and Techniques (also known as academic
mindsets and behaviors) from the Conley (2011) model, a 1-5 Likert survey was sent to 173 8th grade
students within the target middle school at the end of the 2015 school year. This survey had been
previously developed during the 2013/2014 school year (Harm, Austin & Thienes, 2014). Questions
were aligned to the Key Learning Skills and Techniques categories indicated in figure 1 above.
Cronbach’s alpha values were calculated to ensure internal validity of the survey tool with reliability
values of 0.70 sought as “reliable”. Question groups that met this requirement were found to be:
1) Collaboration (n=2; Alpha=.84)
2) Persistence (n=3; Alpha=.71)
3) Technology proficiency (n=5; Alpha=.82)
4) Study skills (n=5; alpha=.75)
5) Goal setting (n=2; Alpha=.64) *
6) Growth mindset (n=5; Alpha=.70)
7) Ownership of learning (n=3; Alpha=.66) *
Value with an asterisk (*) did not reach the standard of .70 to be considered reliable, but were still used
within the analysis due to their proximity to this value. Additionally several transition-type questions
were added to the survey to assess college and career knowledge in order to determine if there were
differences in the academic mindset of students with varying post-secondary expectations.
Results:
Several analyses were completed to analyze and compare survey responses to the career and college
readiness mindset framework questions. Additionally, demographic comparisons were created as and
graphs are provided below as figures 2, 3 & 4.
Figure 2. Racial Demographics
Figure 3. Family education experience- estimate
Figure 4. Career awareness estimate
Figure 5. Perception of skills needed in future career
Mindset survey question analysis.
Results of the student survey show that most students within the target middle school rated their
perceptions as above average in the categories of growth mindset (M=3.84, SD=.69), persistence
(M=3.97, SD=.65), ownership of learning (M=3.70, SD=.83), and technology proficiency (M=3.82, SD=.75).
Students ranked their perceptions of their study skills (M=3.31, SD=.83) and enjoyment of collaboration
(M=3.04, SD=1.07) as signficantly lower than the other four categories of the framework (p > .001).
These two categories were found to be significantly lower than the previous four, indicating that these
responses were not lower by chance, but rather indicate a consistent trend in responses. The values of
study skills and collaboration were not found to be statistically different from each other.
Figure 6. Student mean responses to Key Learning Skills and Technique (mindset) framework questions
When comparing the Key Learning Skills and Techniques survey values between school years (8th
graders from 2014 {n=302} vs. 2015 {n=173}) a signficiant difference in responses was found in the
category of study skills, with an increase in mean scores. Survey values rose from M=3.07, SD=.82 in
2014 to a mean score of M=3.32, SD=.83 in 2015. This indicates a positive increase in effect size of d=.29
or a small to medium effect. Additionally, a signficant difference was found in the category of
collaboration with a decrease in mean values moving from M=3.42, SD=1.11 to M=3.04, SD=1.17. This
difference indicates an effect size of d=-.34 or a small to medium negative effect. All other values were
statistically equal. No value was obtained for growth mindset during the 2014 survey, therefore
comparison could not be made.
Figure 7. Comparison of Key Learning Skills and Techniques survey values by year
Conclusions and Implications.
Based on this year’s survey results which were aligned with the Career and College Readiness category
of Key Learning Skill and Techniques (academic behaviors and mindsets) the highest rated values by
students were in the growth mindset, persistence, ownership of learning, and technology proficiency
categories. The highest rated category by all students was the persistence cateogry. These values could
very well be the result of the ongoing focus on growth mindset, grit, and perseverce that the school has
been working on (Dweck, 2006). This could be an indication that this “academic mindset” work which
the school is doing is paying off and benefitting their students. In the 2015/2016 year, this school is
entering into a 1:1 technology project in which all students will have their own devices in the upcoming
years. It will be of interest to monitor the technology proficiency category through the next few years to
observe any differences in confidence that arise due to familiarity with these tools.
The study skills category which indicates, at least in part, students’ perception of the importance
of studying rose signficantly from the previous year. Again, this could be in part due to the focused
mindset work the school has been accomphishing throughout the year. An increase in growth mindset,
grit, and perseverance- accomanied with a focus on career and college knowledge- would certainly lead
students to seeing the importance of doing well in school and the need to study. This survey may be a
reflection of just this occurrence. One point to mention however, is that this study skills value is still
signficantly lower than the other categories indicating a need for further focus on relevance and
authentic-type assessments which move beyond “the test”. Student perception of “studying” is most
likely confined to what they percieve as preparing for a paper-and-pencil exam and does not include
looking further in depth into information they deem interesting and/or relevant to their lives or simply
making sure they understand the content that they are interacting with. Schools should continually
search for these topics and methods that lead students into deeper learning which may occur even on
their own time- and this could be thought of as studying. The lower value for collaboration also
indicates the need to define and teach effective collaboration and group-work skills which are a
necessity in the 21st century workplace. Though it is acknowledged that this is a difficult age for
effective and efficient group work, this may be all the more reason to focus on effective techniques in
the elementary grades (K-5) as well as the beginning of middle school (6-8). Research-based techniques
could be found which lead to effective collaborative experiences for students. This seems to be a
consistently low category in many schools and is one that needs to be focused upon due to the strong
impact that collaborative learning is know to have and to the explicit need for it in today’s workplace.
Finally, it will be interesting to watch the career and college knowledge categories through time.
As the school continues to implement activities and projects focused on students developing this
knowledge. It is through the knowledge of a career and the clear development and articulation of a
pathway to get there (ie. what courses will you need to take to enter that profession) that students will,
at least in part, see the relevance to the courses they are taking. As schools move toward a focus on
building strong academic behaviors and mindsets, they will be equiping their students with the career
and college readiness skills which will allow them to solve problems, work with others to accomplish
large tasks, and find and understand information as needed to accomplish them. These students will
then truly be career and college ready.
Work Cited
Dweck, C.S. (2006). Mindset: the new psychology of success. New York: Random House.
Farrington, C.A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T.S., Johnson, D.W., & Beechum, N.O.
(2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping
school performance: A critical literature review. Chicago: University of Chicago Consortium on
Chicago School Research.
Harm, J.E., Austin, L., Thienes, K. (2014). Career and College Readiness- Middle School Key Academic
Behaviors Survey Pilot Study.
http://www.westada.org/site/default.aspx?PageType=3&ModuleInstanceID=50524&ViewID=7b
97f7ed-8e5e-4120-848f-a8b4987d588f&RenderLoc=0&FlexDataID=21861&PageID=20734