www.legacy.ac.uk
Professor Christina Hughes
Sheffield Hallam University
LEGACY - Learning and Employability
Gain Assessment CommunitY
• 18 Russell Group
Universities
• Birmingham, Bristol,
Durham, Cambridge, Exeter,
Imperial, Kings, Leeds,
Liverpool, Manchester,
Newcastle, Nottingham,
Queen Mary, Sheffield,
Southampton, UCL,
Warwick, York
• HEFCE Pilot Programme
• 3 Years
– Oct 2015 – Sept 2018
• 4 Work Packages
– Measurement of
Learning Gain
– Employability
• Higher Education
Timeline
Year 1 2015/16
Year 2 2016/17
Year 3 2017/18
Measurement of Learning Gain WP1
R2 Strengths WP2
Career Adaptabilities WP3
International Experience & Employability WP4
What is learning gain?
• Distance travelled (HEFCE)
• The improvement in knowledge, skills, work-readiness, and
personal development made by students during higher
education (RAND, 2014).
• The change (potentially, progress) in the knowledge, skills,
and competencies that are relevant across disciplines made
by students during higher education (Vermunt, Vignoles & Ilie,
2016).
4
Which dimensions are we measuring
in LEGACY?
• Cambridge –
– cognitive, meta-cognitive, affective, sociocommunicative
– plus cross-cutting dimensions – openness,
research, moral
• Warwick – Realise 2 Strengths
• Birmingham – International experience
• Nottingham – Career Adaptabilities
Cambridge: Knowledge, skills, and
competencies
Cognitive
Critical thinking
Analytical
thinking
Cognitive
abilities
Synthesising
Analysing
Evaluating
Problem solving
Meta-cognitive
Self-regulation
Life-long learning
attitude and
motivation
Learning to learn
Need for
cognition
(information
seeking)
Affective
Sociocommunicative
Attitudes towards
own discipline and
learning/studying in
general
Levels of belonging
in social learning
networks
Motivation
Social
embeddedness
Engagement
Professional and
academic interest
Communication skills
6
Cambridge: Cross-cutting dimensions
Openness
dimension
Open-mindedness; View of intelligence;
Epistemological stance
Research
dimension
Curiosity; Interest in research ; Interest in
knowledge; Attitude to sharing ideas
Moral
dimension
Moral reasoning
7
WARWICK: STRENGTHS
Theoretical underpinning
•
Base-Positive Psychology
•
R2 Strength
Evidence base
•
Strengths-based recruitment is increasing
•
Students struggle to demonstrate self-awareness so can ‘undersell’
•
Strengths awareness helps develop supporting, authentic evidence
Method
•
Training in R2 Strengths of careers professionals
•
Two cohorts of students – one engaged in full R2 programme; the other
in strengths identification only.
Birmingham: Internationalisation
• Impact of international experience on students’ employability
(Wright and Jones, 2014)
•
Study abroad
•
Work abroad
•
On-campus experiences
• How can internationalisation enhance students’ employability
skills?
• Method:
•
Semi-structured interviews
•
pre and post experience
•
Identify self-perceived employability learning gains
Birmingham: Internationalisation
Study Abroad:
• Motivation: academic related, employability related,
personal interest & development
Work abroad
• Extrinsic factors (CVs, experience)
• Intrinsic motivations
• Gain global competencies (Diamond et al, 2011)
•
On-campus experience
• Risk averse behaviour
• Perceived positive impact
Nottingham: Career Adaptability
‘The capability of an individual to make a series of successful
transitions where the labour market, organisation of work and
underlying occupational and organisational knowledge bases may be
subject to considerable change’ (Bimrose et al, 2011)
• Stages of Careers Adaptability
Source: Frigerio and Wright (2014)
Adaptive
Readiness
Adaptability
Resources
Adapting
Adaptation
as outcome
• Four Career Adapt-abilities (Savickas, 1997)
• CAAS-International validated in 13 countries (Savickas &
Porfeli, 2012)
NOTTINGHAM: CAREER ADAPTABILITIES –
INITIAL FINDINGS
Career Adaptability Scores
Careers questions
Mean concern
Mean control
Mean curiosity
Mean confidence
Mean overall adaptability
15% (N=64) have spoken to careers
about ideas/plans
27% (N=116) have attended a careers
fair
12% (N=50) have attended a career
skills workshop
16% (N=67) have had a position of
responsibility
32% (N=137) have done a CV
26% (N=111) have some work or
volunteering experience
17.9
21.8
19.0
21.1
80.0
 Means are very similar to the means
found in the original UK study
 Score ranges showed a normal
distribution with varying degrees of
skewness to the right)
How does the work of LEGACY
contribute to teaching excellence?
Empirical Evidence of the Validity of a Range of Models to Measure Learning
Gain, via
– A longitudinal explanatory model of student learning gains by a range
of personal and environmental factors within disciplinary and
employability domains;
– Analysis of core variables related to widening participation, gender,
ethnicity, tariff entry scores and so forth
• Empirical assessment of a range of tools developed to measure learning
gain, and establishment of their validity and reliability;
• A better understanding of the potential to measure learning gain at scale;
• Knowledge Exchange and Critical Awareness of debates and issues
related to the measurement of learning gain – check www.legacy.ac.uk
;
14
Will there by one way to measure
learning gain?
• Our evidence, and that coming out of the overall programme, is
that the complexities and dimensions of what is measured required
a range of tools and techniques.
• All measures have limitations ie measuring employability gain
based on credentials or skills provides measures of different
dimensions
• All measures have limitations in measurement (cf self-reporting and
random control trials)
• We need to critically explore the notion of whether there is a holy
grail of one instrument/one measure that provides a common point
of comparison across subjects and institutions
• We need to ask what measures are measuring – are they in effect
proxies for something else that becomes invisibilised
• We should be measuring what we value not valuing what we are
required to measure.
Find out more...
Programme
Lead:
Project
Manager:
Research
Fellows:
Professor Christina Hughes
[email protected]
Sunil Maher
[email protected]
Dr. Heike Behle
[email protected]
Dr Toni Wright
[email protected]
Dr. Sonia Ilie
[email protected]
Research Programme Leaders:
Cambridge: Professor Jan Vermunt [email protected]
Professor Anna Vignoles [email protected] &
Birmingham: Eluned Jones [email protected]
Nottingham: Dr. Nalayini Thambar [email protected]
Warwick:
Anne Wilson [email protected]