ACA Navigators in
Illinois: An Analysis
of Training Efficacy
November 2014
Presented By: Michele Thornton, MBA
Dr. Elizabeth Calhoun
Dr. Lindsey Leininger
Introduction
• The Affordable Care Act (ACA) created a new workforce
of insurance Navigators tasked with providing low- and
moderate- income populations with unbiased,
linguistically & culturally appropriate information
regarding coverage options.
• Our training team designed and delivered a three-day
curriculum to over 1,500 Navigators & Assisters across
Illinois.
• The objective of this study is to assess the efficacy of the
training.
Illinois government
requires certification
• In August of 2013, Governor Quinn signed a bill
specifying certification requirements for Navigators and
In Person Assisters. These entities must “successfully
complete the federal and state training provided by the
exchange.”1
Requirements:
• Federal Training – Approximately 5-20 hours online
• State Training – 3 days in-person/online blended
• Pre-Post Assessment
• Identification Verification
• Background Check
• Disclose any Conflict of Interests
• Continuing Education to Recertify Annually
1
Illinois General Assembly. Public Act 98-0524. August 23, 2013.
State curriculum enhances
federal online training
Online & Day One
Day Two
ACA Overview (online)
The Marketplace
Responsibilities (online)
Employer Insurance
Ethics (online)
Special Populations
Core Eligibility
Post-Enrollment
Medicaid
Communication
Insurance 101
Outreach
Robust training methods
• Traditional Lecture
• Case Studies
• Observational Learning
• Game/Activity Based
• Role Playing
• Asset Mapping Exercise
• Networking and Peer Education
Research aims
1. To characterize the variation in baseline levels of
knowledge among the socioeconomically heterogeneous
Navigator population.
2. To assess subject matter mastery following training,
with a specific focus on the performance of subgroups
demonstrating lower levels of baseline knowledge
3. To gather Navigator preferences regarding ongoing
continuing education, with the goal of informing future
curriculum development and other training activities
Data & Methods
•
The result of interest is the score on a 62 item
assessment administered pre- and post- training.
•
We identify sociodemographic subgroups (age, gender,
race/ethnicity, education, experience, organization
type) along which we expect to find differential impacts
of the training curriculum on learning.
•
We test for these effects using standard multivariate
OLS regression techniques.
Navigator population
Initial Sample of Training Participants: 1518
Total Number that go on to become “Certified”: 1250
n
%
Gender
n
%
No College
157
10.3
470
461
429
31.0
30.4
28.3
760
412
346
50.1
27.1
22.8
Education
Female 1222
80.5
Male
294
White
Hispanic/Latino
Black
626
387
299
Some College
College Degree
41.2
Post College
25.5 Years Work Experience
19.7
0 to 5
Asian
Other
86
118
5.7
7.8
Race/Ethnicity
19.37
6 to 15
16 +
Comparison of navigator race/ethnicity to percent
of uninsured population in IL2
45%
41% 41%
40%
33%
35%
30%
26%
25%
19% 20%
20%
15%
10%
5%
5%
6%
0%
White
Hispanic/Latino
Uninsured
2United
Black
Navigators
States Census Bureau. Health Insurance Coverage Status. 2012
American Community Survey 1 Year Estimates. Published June 27, 2013.
Accessed September 18, 2014.
Asian
Navigator population
Initial Sample of Training Participants: 1518
Total Number that go on to become “Certified”: 1250
n
%
Organization Type
n
%
Insurance Type
Healthcare Provider
362
23.9
Employer
867
57.1
Community Org
674
44.4
Government
Other
285
197
18.8
13.0
Medicare
Medicaid
46
133
3.0
8.8
Private Individual
177
11.7
Uninsured
199
13.1
Other
94
6.2
Assessment results
show proficiency
• The overall change in assessment test scores
from Mean Pre (70.97%) to Mean Post (91.92)
was 20.95 percentage points.
• This represents a 29.5% increase over the
Mean Pre Score.***
*** p<.001
** p<.01
*p<.05
Regression Adjusted Mean
by Subgroup (Black)
80% Passing Threshold
100.00
90.00
80.00
70.00
60.00
50.00
40.00
30.00
20.00
10.00
0.00
92.35
90.16
71.77
67.68
Black
Pre***
*** p<.001
** p<.01
*p<.05
All Other
Post***
Regression Adjusted Mean
by Subgroup (Uninsured)
80% Passing Threshold
100
90
80
70
60
50
40
30
20
10
0
92.08
90.82
71.48
67.54
Uninsured
Pre***
*** p<.001
** p<.01
*p<.05
All Other
Post*
Regression Adjusted Mean
by Subgroup (No College)
100
90
80
70
60
50
40
30
20
10
0
80% Passing Threshold
92.01
91.14
71.20
68.98
No College
All Other
Pre**
*** p<.001
** p<.01
*p<.05
Post
Conclusions: Navigators
positioned for success
• Although the material was initially unfamiliar and the
population heterogeneous, the training program
brought all identified sub-groups from a below passing
mean score (71%) on Pre-Test up to above
proficiency in a Post-Test Assessment.
• Some significant differences persist after training,
even above proficiency – may make sense to assess
cultural sensitivity of training.
• The composition of outreach workforce closely
parallels demographics of uninsured. This combined
with competencies gained through enhanced training
suggests a high likelihood of providing expertise, as
well as culturally & linguistically appropriate services.
Future Directions
•
•
A post-training follow up survey was administered – with
62% response rate.
•
Navigators across demographic highlight Medicaid &
Marketplace plan and policy information as most
important for continuing education.
Matching data to determine if assessment scores make a
difference in enrollment performance.
Future Directions
•
Additional planned analysis of survey results includes
identifying best practices & organization level characteristics
associated with navigator enrollment success.
•
We believe that our results are instructive both for future
navigator-like efforts in addition to ongoing workforce training
efforts seeking to target other domains of health promotion.
Acknowledgements
We would like to thank the Illinois Health Insurance
Marketplace team (Get Covered Illinois) and our
community partners (Shriver Center, MidAmerica Center
for Public Health Practice), DuPage Federation for Human
Rights) for their support and collaboration on this project
Contact Information
•
•
•
•
•
Michele Thornton, MBA
University of Illinois – Chicago
P. 708-606-6060
E. [email protected]
T. @ChicagoHCTweets