Implementing Language Access Plans:
What Works? What Counts?
Evaluation Results
Azadeh Khalili
October 5, 2011
1
Table of Contents
1.
2.
3.
4.
Evaluation Overview
Quick Snapshot of Participant Ratings
Results from the Morning Session
Results from the Afternoon Breakout
Sessions
5. Considerations and Implications for Next
Steps
2
Evaluation Overview
• AECF evaluation forms measured extent to which convening achieved 5
desired results:
1.
2.
3.
4.
5.
•
Also asked:
1.
2.
3.
4.
•
Understand how language access contributes to achieving agency’s mission
Identify strategies and action steps to move language access forward
Connect with colleagues from other agencies to learn and share ideas
Use knowledge, skills, and networks gained today to strengthen my
language access work
Know who to contact for TA going forward
Effectiveness of morning speakers and roundtable discussions
What participants found useful
What would have been more useful
Other comments
Separate evaluations of morning session (116 respondents) and
afternoon breakouts (77 respondents)
3
Snapshot of Participant Ratings:
The Convening Was Productive Use of Time
Morning Session
Concurrent Afternoon Breakout Session A
Concurrent Afternoon Breakout Session B
0%
strongly agree
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
agree
neutral
disagree
strongly disagree
4
Results from the Morning Session
5
Results of Morning Session
Connected With Colleagues
Will Use Knowledge, Skills, and Networks
The Speakers Enhanced My Thinking
Roundtable Discussions Enhanced Thinking
Know Who to Contact for TA
Identified Strategies and Action Steps
Understanding of LA & Agency Mission
0%
strongly agree
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
agree
neutral
disagree
strongly disagree
6
What was most useful about the
morning?
Note: Relative size of words reflects how often they were mentioned in participant responses.
7
What would have made the morning
more useful?
Note: Relative size of words reflects how often they were mentioned in participant responses.
8
Additional comments from the
morning session…
Note: Relative size of words reflects how often they were mentioned in participant responses.
9
Results from Afternoon Breakout Sessions
10
Aggregate Results of Afternoon Breakout Sessions
Session A (n=76)
Productive Use of Time
Learned New Skills to Move LA Forward
Connected with Colleagues
Helped Refine Action Plan
Session B (n=64)
Productive Use of Time
Helped Refine Action Plan
Connected with Colleagues
Learned New Skills to Move LA Forward
0%
strongly agree
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
agree
neutral
disagree
strongly disagree
11
Results of Each Breakout Session
Meeting Translation Needs B (n=3)
Meeting Translation Needs A (n=20)
Interpretation: Outside Vendors B (n=9)
Best Practices in Contracting A (n=10)
Interpretation: Testing & Training B (n=18)
Interpretation: Testing & Training A (n=16)
Best Practices in Contracting B (n=10)
Monitoring and Evaluating Plans A (n=19)
Interpretation: Outside Vendors A (n=10)
Monitoring and Evaluating Plans B (n=24)
0%
strongly agree
* Results are the aggregate
connected with colleagues.
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
agree
neutral
disagree
strongly disagree
responses for the 3 desired results: learned new skills; helped refine action plan; and 12
Results of Concurrent Breakout #1:
Meeting Your Agency’s Translation Needs
Session A (n=20)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
Session B (n=3)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
0%
strongly agree
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
agree
neutral
disagree
strongly disagree
13
Results of Concurrent Breakout #2:
Interpretation Testing and Training
Session A (n=16)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
Session B (n=18)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
0%
strongly agree
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
agree
neutral
disagree
strongly disagree
14
Results of Concurrent Breakout #3:
Interpretation Using Outside Vendors
Session A (n=10)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
Session B (n=9)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
0%
strongly agree
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
agree
neutral
disagree
strongly disagree
15
Results of Concurrent Breakout #4:
Best Practices in Contracting
Session A (n=10)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
Session B (n=10)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
0%
strongly agree
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
agree
neutral
disagree
strongly disagree
16
Results of Concurrent Breakout #5:
Monitoring and Evaluation
Session A (n=19)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
Session B (n=24)
Learned New Skills to Move LA Forward
Helped Refine Action Plan
Connected with Colleagues
Productive Use of Time
0%
strongly agree
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
agree
neutral
disagree
strongly disagree
17
Comments from afternoon breakout
sessions
Note: Relative size of words reflects how often they were mentioned in participant responses.
18
Considerations and Implications for
Next Steps
• Design team set very ambitious goals for a single
day with a large, heterogeneous group
• High demand among participants for continued
engagement:
1.
2.
3.
4.
Examples of DOJ-approved LAPs
Peer learning from federal agencies linked to specific
elements/challenges of their own LAPs
More intentional networking (eg, all law-enforcement, mixmax from multiple agencies) with more choice of content
Responsive to concerns re: logistics (eg, room, handouts, etc)
• Opportunities to leverage greater role for federal
LA task force members
19