Implementing new health and
K–12 education indicators in national and
international assessment systems
Patrick Kyllonen
Center for Academic and Workforce Readiness and Success
Educational Testing Service
Princeton, NJ, USA
Understanding the opportunities for collaboration between the
health and education sectors in a culture of health
Cinco de Mayo, 2016
Outline
• Benefits of Schooling
• Schools Develop Social Emotional Skills (SES)
• Frameworks for SES Development
• Measurement Approaches for Tracking SES Development
2
2
Benefits of Schooling
• Education benefits individuals and society
̶ economic growth, less unemployment, better jobs, less crime, higher civic
engagement (human capital theory)
• What is it about education that produces these benefits?
̶ Cognitive skills — mathematics, language, problem solving — but cognitive
skills are only part of the benefit
̶ Other skills — punctuality, work ethic, getting along with others, impulse
control, listening to the teacher — may be as or more important
3
Copyright © 2016 by Educational Testing Service. All rights reserved. ETS, the ETS logo, MEASURING THE POWER
OF LEARNING and SUCCESSNAVIGATOR are registered trademarks of Educational Testing Service (ETS). 36320
3
• Growing recognition that schools do more than teach mathematics and
language
4
4
Singapore
55
Social and emotional skills
• are the most important result of early childhood interventions
designed to boost IQ (Barnett, 2011; Heckman)
• predict grades throughout K–16 as strongly as IQ does
(Poropat, 2009)
• account for as much or more of the educational attainment
effect as cognitive skills do (Bowles et al., 2001)
• predict mortality, divorce, occupational attainment (B. Roberts et al.,
2007), life satisfaction, crime record, income level, physical
health, and parenting skill (Moffitt, Poulty, & Caspi, 2013)
6
6
Which skills?
7
2008
2010
2011
2012
2016 (forthcoming)
Intrapersonal and interpersonal skills in higher education
8
Farrington, et al – University of Chicago Consortium on Chicago School Research (2012)
• Academic Behaviors
̶
Going to class, doing homework, organizing materials, participating, studying
• Academic Perseverance
̶
Grit, tenacity, delayed gratification, self-discipline, self-control
• Academic Mindsets
̶
“I belong in this academic community” (sense of belonging; stereotype threat)
̶
“My ability and competence grow with my effort” (attribution theory; theories of
intelligence; locus of control)
̶
“I can succeed at this” (self-efficacy)
̶
“This work has value for me” (expectancy-value theory; see also, TPB)
Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The
role of noncognitive factors in shaping school performance: A critical literature review. Chicago: University of Chicago Consortium on Chicago School Research.
9
9
CASEL: Social Emotional Learning (SEL)
The ability to take the
perspective of others,
particularly from different
backgrounds and cultures
Social
Awareness
Self
Management
SEL
The ability to establish
Self
Relationship
and maintain healthy
Awareness
Skills
relationships with others
through communication,
listening, cooperating,
Responsible
negotiating, and helpDecision
seeking
Making
The ability to regulate
one’s emotions and
behaviors and to set
and pursue personal
and academic goals
The ability to
recognize one’s
thoughts and
emotions and
their relationship
to behavior
The ability to make constructive choices
about behavior and social interactions
10
10
Worldwide Interest in the Development of Social and Emotional Skills
Phase 1
•
Expert panel: Economists,
psychologists, educators
•
Literature review, conceptual
framework
•
High-level policy forum, Sao Paolo,
Brazil 2014
Phase 2
•
International longitudinal study of
skills development in cities
•
Grades 1, 7 (ages 6, 12)
•
Students, teachers, parents
•
Learning contexts, cognitive, social
and emotional skills, outcomes
•
Launch: 2019
March 10, 2015
http://www.keepeek.com/Digital-Asset-Management/oecd/education/skills-for-social-progress_9789264226159-en#page1
11
11
OECD Skills Framework
Contextual factors: Schools, Family, Community and Workplace
Basic cognitive
ability
• Pattern recognition
• Processing speed
• Memory
Knowledge ̶
acquired
•Access
•Extract
•Interpret
Knowledge ̶
extrapolated
• Reflect
• Reason
•Conceptualise
Cognitive
• Mental capacity to acquire
knowledge, thoughts
and experience
• Interpret, reflect and
extrapolate based on the
knowledge acquired
Social and
Emotional
Achieving goals
• Perseverance
• Self-control
• Passion for goals
Individual capacities that (a) are
manifested in consistent patterns of
thoughts, feelings and behaviours,
(b) can be developed through
formal and informal learning
experiences, and (c) influence
important socioeconomic outcomes
throughout the individual’s
life.
Working with
others
• Sociability
• Respect
• Caring
Managing
emotions
• Self-esteem
• Optimism
• Confidence
Outcomes: Education, workforce, civic, safety, health and well-being, environment
12
OECD’s Skills for Social Progress
Policy Messages
• Children need a well-balanced set of cognitive, social and emotional skills (SES)
for life success and societal progress
• Children’s capacities to achieve goals, work effectively with others and manage
emotions help improve lifetime outcomes
• SES can be raised by improving learning environments and mobilizing
intervention programs
• “Skills beget skills” and early investment in SES is key to improving life prospects
• Regular SES assessments provide valuable information to improve learning
contexts and ensure they are conducive to skill development
• Policymakers acknowledge the importance of SES, but the level of policies and
programs varies
13
13
Methods
14
3 Categories
• Administrative records
̶ Absenteeism, tardiness
̶ Grades (Kautz & Heckman)
• Survey methods
̶ Self and other ratings
̶ Anchoring vignettes
̶ Forced choice
• Performance tasks and games
̶ Collaborative problem-solving example
̶ Emotional intelligence example (DANVA)
15
15
ETS’s SuccessNavigator® Battery
Organization
Academic Skills
Meeting Class Expectations
Sensitivity to Stress
Determination
Self-Management
Academic Self-Efficacy
Test Anxiety
Connectedness
Social Support
Institutional Support
Barriers to Success
Commitment to College
Commitment
Institutional Commitment
I make a schedule for getting my school work done.
I take due dates seriously.
I attend almost all of my classes.
I complete the reading that is assigned to me.
I get stressed out easily when things don't go my way.
I am easily frustrated.
When feeling stressed about the amount of homework I have, I try to
get organized to get on top of my homework.
When worried about a test, I get prepared regardless.
I'm confident that I will succeed in my courses this semester.
I can do well in college if I apply myself.
When taking a test, I think about what happens if I don't do well.
Before a test, my stomach gets upset.
I feel connected to my peers.
People understand me.
If I don't understand something in class, I ask the instructor for help.
I know how to find out what's expected of me in classes.
Family pressures make it hard for me to commit to school.
People close to me support me going to college.
One of my life goals is to graduate college.
The benefit of a college education outweighs the cost.
This is the right school for me.
I’m proud to say I attend this school.
16
Single Statements Rating Scale
Strongly
disagree
Disagree
Agree
Strongly
agree
Please indicate your answer to each item by clicking on the appropriate circle
1.
I keep my promises
⃝
⃝
⃝
⃝
2.
I am generally pretty forgiving
⃝
⃝
⃝
⃝
Forced-Choice
For each pair of statements please click on the one that is most like you
1.
I keep my promises
⃝
2.
I am generally pretty forgiving
⃝
Drasgow, Stark, Chernyshenko, Nye, Hulin, & White (2012).
18
Other examples
• “I think of others first”
• “I am generally prepared”
• “I feel comfortable around people”
• “I wait for my turn”
• “I sense other’s wishes”
• “I make plans and stick to them”
18
19
ST61
01 Below you will find descriptions of three mathematics
teachers. Read each of the descriptions of these teachers.
Then let us know to what extent you agree with the final
statement.
(Please check only one box on each row.)
Strongly
agree
a) Ms. Anderson assigns mathematics homework
every other day. She always gets the answers back
to students before examinations. Ms. Anderson is
concerned about her students’ learning.
b) Mr. Crawford assigns mathematics homework once
a week. He always gets the answers back to
students before examinations. Mr. Crawford is
concerned about his students’ learning.
c) Ms. Dalton assigns mathematics homework once a
week. She never gets the answers back to students
before examinations. Ms. Dalton is concerned
about her students’ learning.
1
1
1
Agree
2
2
2
Disagree
Strongly
disagree
3
4
3
4
3
4
3
4
02 My teacher lets students know they need
to work hard.
1
2
19
20
ST61
Agree Disagree Strongly
disagree
-
-
Mary is kind to others most of the time. She sometimes
speaks out of turn, but usually is able to control her
emotional reactions, even when intentionally provoked by
others. Mary has a high level of “self control.”
I have a high level of “self control.”
5th grade
6th grade
1
2
3
4
1
2
3
4
20% of the students rate themselves at or above Mary
35% of the students rate themselves at or above Mary
20
21
Situational Judgment Tests (SJTs)
“Teamwork & Collaboration”
As part of a class project, you serve as a volunteer for a nonprofit
agency. In a discussion about how to find new volunteers, you bring up
what you think is a great new idea. But the others tell you that the
idea is “off base” and not workable. How would you handle this
situation?
Best Worst

 Drop your idea because the group is probably right.

 Point out several good reasons why your idea might work.

 Drop your idea for now, but tell it to your boss later.

 Tell the others that lots of people don’t recognize great
ideas at first.
21
22
23
22
Happy?
Or
Angry?
24
Conclusions
• Growing interest
• Good frameworks
• New (better) methods
24
Copyright © 2016 by Educational Testing Service. All rights reserved. ETS, the ETS logo, MEASURING THE POWER
OF LEARNING and SUCCESSNAVIGATOR are registered trademarks of Educational Testing Service (ETS). 36320
25