Comparative assessment of
skills in Russia: An economic
rationale
T. Scott Murray
DataAngel Policy Research Inc.
[email protected]
What is old is new:
In the beginning: The US
• Key scientific advances:
– Theory: The Abrams main battle tank and
insight into what makes adult reading tasks
difficult. Kirsch and Mosenthal
– Applied statistics: The development of
statistical techniques to summarize
proficiency and to estimate errors. Bock and
Rubin
– Enabled the 1985 Young Adult Literacy
Survey (YALS) ETS,NCES and USDOL
In the beginning: Canada
• In Canada 25 years of Scott measuring
every kind of social and economic ill and
being able to explain very little of it
• Celtic allergy to empires built on unproven
assertions of systemic discrimination
• The Southam survey “24% of adult
Canadians had literacy skill problems”
• Conduct of the survey of 1989 Literacy
Skills Used in Daily Activities (LSUDA)
Precipitated multiple rounds of
data collection:
•
•
•
•
IALS 1994 9 countries
IALS 1996 5 countries
IALS 1998 12 countries
Vanuatu, Immigrants in Ontario, the Deaf and
Hard of Hearing in Ontario
• ALL 2003
• ALL 2005
• PIAAC 2011
What was learned for policy:
• Large differences in skill existed both
within and between countries
• These differences were far larger than
implied by differences in educational
attainment
• The differences mattered to individual
outcomes but were too large to attribute to
differences in educational quality
The policy response:
• There is no problem “Our problem is the
same as that of our trading partners”
• There is a problem but it doesn’t matter, “It
doesn’t increase the size of the economic
pie it is just an allocative mechanism”
• There is a problem, it matters to growth
but it is not our problem “It is a problem for
individuals and firms to correct”
The hypothesis of market failure:
• Only governments have the tools to
correct market failures:
– Information
– Incentives
Issues that need attention: The
dreaded mastery level
• All adult assessments have used 80% as a
proxy for what employers expect of workers
• PISA uses 62.5% because education systems
are not focused on mastery
• OECD is pushing to have the PIAAC standard
changed to distance it from IALS, a step that
would break the series
• Empirical analysis by Somers & Murray
suggests that the Response Proficiency should
vary by occupation from 40% to 95%
Brain Structure
B
a
c
k
to
F
r
o
n
t
brain from
the top
Left to Right
Brain Structure
B
a
c
k
to
F
r
o
n
t
Information
(processing-based)
Knowledge
(thinking-based)
Brain Structure
brain from
the side
What is this brain doing?
Decoding
PET Scan
Brain Structure
What is this brain doing?
Hearing
PET Scan
Brain Structure
What is this brain doing?
Speaking
PET Scan
Brain Structure
What is this brain doing?
Dr. Michael O’Boyle images
from Texas Tech. University
School of Medicine, Department of Neurology
Math
PET Scan
Brain Structure
What is this brain doing?
Thinking
PET Scan
Brain Structure
http://www.nil.wustl.edu/labs/raichle/
Issues for further study:
• Developing a better understanding of
supply, demand and the efficiency of
markets
• Our analysis suggests that supply-side
interventions are not enough. One needs
to manage demand and improve the
efficiency of markets that match workers
with jobs
Theoretical Framework: a “Markets” Model of Skill
Skill Demand
Markets
for
skill:
Skill Supply = skill
stock + net skill flow
from lifelong, lifewide learning
•Education
•Labour
•Health
•Social
Outcomes
Context
• Economic
• Social
• Educational
• Health
MICRO
MESO
MACRO
(Individuals)
(Social Institutions)
(Systems)
+ quality of early
childhood experience
+ quantity of primary
and secondary
education
+ quantity and quality
of tertiary
+ quantity and quality
of
adult learning (formal,
non-formal, informal)
+/- immigration
+/- emmigration
- skill loss associated
with insufficient
demand
+/- social demand for
skill
+/- economic demand
for skill
Skill gain and loss, the behaviour of firms
and the possibility of multiple equilibria:
Document Literacy Score
Canada
300
275
IALS 1994
ALLS 2003
250
225
ALLS 2003 - All Adults
ALLS 2003 - Recent Immigrants excluded
200
20
30
40
50
60
Age in 1994
29
39
49
59
69
Age in 2003
One way in which firms compensate for a workforce with weak literacy
skills is to adopt less productive work organizations, work processes and
technologies of production:
The production frontier: the point
where one gets the most output
at the lowest unit cost
•
Cost
A less productive
configuration
•
Output
Correcting the market failure:
• Projecting down: skill profiles for small
areas
• Projecting down: Skill profiles for detailed
occupations and cost-benefit analyses
• Projecting out: Forecasts of skill supply
and demand
• RCT’s and quasi-experimental studies to
establish effect sizes and cost-benefit
• Our basic hypothesis: A failure in the literacy market
• The literacy market failure can be traced back
to:
– Complacency due to 50 years of economic
success
– A failure to appreciate the implications of the
changes occurring in the global economy
– A lack of information on the nature of the problem
– A lack of tools to assess skills
– A lack of efficient and effective instructional
programs
Projecting down: The local
distributions of skill
Alberta Number of Adults at Levels 1 and 2 Prose Literacy
0 - 30
31 - 60
61 - 90
91 - 120
121 - 150
151 - 180
181- 210
211 - 240
241 - 270
More than 270
No Data
175
87.5
0
175 Kilometers
Projecting down: Absolute and relative risks of
being in literacy shortage by occupation,
Canada 2006:
Colleges are admitting large numbers of
low skilled youth:
Percent of Population at Levels 1-3 Enrolled in Collegeby Province - by Literacy Level
Ontario
Newfoundland and
Labrador
Alberta
Quebec
Nova Scotia
British Columbia
New Brinswick
Saskatchewan
Prince Edward Island
Manitoba
0%
10%
20%
Level 1
30%
Level 2
40%
Level 3
50%
60%
Fixing the flow: Quality of secondary graduates is not
improving, large percentages have skills below those
needed to take full advantage of PSE, and there are
large social inequities
Projecting out: How the distribution of performance by
level is likely to change 2001 - 2031
Population
Population Projections - Population at various
Literacy Levels Province (All) - Immigration (All) - Age
Group (All) - Education Level (All)
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Level 4/5
Level 3
Level 2
Level 1
2001
2005
2015
2031
Literacy Level
59-B.C.
0%
-1%
1%
-1%
1%
Literacy demand rising rapidly and literacy supply flat:
Shortages will grow over coming decade
Understanding the learning needs of lowskilled adults: The transition from
learning to read to reading to learn:
Learning to read
Reading to learn
Proficiency dominated by cognitive strategies
Proficiency dominated by
mechanics of reading
0
225
Level 1
275
Level 2
325
Level 3
375
Level 4
500
Level 5
Literacy market segments in Canada:
Segments in the Canadian
literacy market:
•
The efficiency of the market for literacy: Balance of total supply and total
demand (in points)
The efficiency of the market for literacy: Literacy skill
surpluses and shortages by level of the job (in workers)
Figure 4.3A Number of workers in literacy skill surplus
and shortage by literacy proficiency level, all
occupations, 48-Alberta, 2006
Skill
shortage/surplu
s (workers)
600,000
400,000
200,000
-
-200,000
1
2
3
-400,000
-600,000
Proficiency level
4
5
50% of workers in Alberta are in
shortage:
Estimated benefits are clear:
Issue for further study: Improving our
understanding of how skill drives
productivity and wages
• Two competing hypotheses:
– Skill is a general driver of productivity growth
across the whole spectrum of occupational
demand
– Skill only pays in jobs where demand for use
is high
– Reder: Skill pays but only in bad times
33% of wage variation in Canada is
attributable to skill differences:
The impact of skill on individual labour market outcomes
Earnings and literacy proficiency, controlling for education
and labour force experience
… literacy explains a significant fraction of wage variability in Portugal,
but less than education or experience …a sign of an inefficient labour market
Standardised regression w eights x 100
or low demand…
0
Countries are
ranked by the
magnitude of the
effect parameter
associated with
educational
attainment
10
20
30
40
50
C anada
PRC
F inland
Unit ed Kingdo m
N o rway
N ew Z ealand
A ust ralia
Unit ed St ates
Ireland
P o rtugal
H ungary
D enm ark
Slo v enia
N etherlands
B elgium (F landers )
Swit zerland
Sweden
C hile
C zech R epublic
Germ any
P o land
Educational attainment
Literacy prof iciency
Experience
Source: International Adult Literacy Survey, 1994-1998.
A Framework for Thinking about Essential
Skills at the Individual Level:
Mechanisms
• Reliance on practical and
crystallized cognitive tools
• Reliance on others
• Avoidance
Changing
Skill Demands
• Tasks to be
accomplished/
imposed by
society/economy
•Changes
associated with
the life course
Resolution
Processes
Individual
Outcomes
(MICRO)
• Functioning
in socially
heterogeneou s
groups
• Acting
autonomously
• Individual
goals/aspirations
Reliance on Physical tools
• Reliance on fluid and creative cognitive
and meta-cognitive tools
Outcomes
for
Social
Institutions
(MESO)
• Successful
institutions
•firms
•families
•communities
•schools
Societal
Outcomes
(MACRO)
• A well
functioning,
equitable
economy and
society
Issues for further study: The impact of
skills on health outcomes and costs:
• Adults with low health literacy scores
remain 2.5 times more likely to be in fair or
poor health even after adjusting for a wide
range of variables
• Elimination of skill shortages using “best
practice” interventions would break even if
it saved 1% of health costs and would
yield large economic benefits
Issues for further study: The impact of
skills on health outcomes and costs:
• Need to understand biochemical pathway
that links skill to health
• Need an RCT to know
• Hypothesis:
– Low skill = bad outcomes = chronic stress =
high blood cortisol =impaired immune
response = higher rates of CVD, diabetes and
cancer
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Qatar
Russia
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Qatar
Russia
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Assessment the old way:
•
•
•
•
Technically demanding
Costly
Operationally demanding
Slow
Assessment the new way:
• web-based
• adaptive
• multi-purpose,
• low cost
• Immediate results
for:
–
–
–
–
–
Prose literacy
Document literacy
Numeracy
Reading components
Oral fluency