Electric Paper Ltd  10 Greycoat Place  London  SW1P 1SP  020 7960 6046
Student Surveys - Best Practice
Student Surveys - Best Practice
Eric Bohms
“If you can't measure it, you can't manage it”
Peter Drucker – Drucker Institute
Student Surveys - Best Practice
Session Agenda:
1.
Best Practice Survey
2.
Paper vs. Online: the ongoing debate
3.
Best practices for HE survey administration
4.
Ideas for driving cost and efficiency saving
5.
Why surveys? The ‘Audit’ culture vs. the ‘Feedback’ culture
Student Surveys - Best Practice
We asked: What survey methodologies are
used at your institution?
9% Paper only
9% Online only
12% Hand-Key &
Online
3% Hand tabulation
77% Paper & online
Population: UK Quality Managers and University Administration staff at 58 institutions 2010
Paper vs. Online
Instant feedback
Low deployment cost
High response rate
Easy to administer
Easy for the student
Location independent
Advantage
Disadvantage
Survey Administration
5
Printing costs
Low completion rates
(statistically significant)
Hand writing
Strong opinions likely
The Debate Continues...
Online
Surveys
Paper Data
Capture
Add-on
Survey
6
Hybrid
Outsourcing
Things to Consider When Managing Surveys in
Higher Education: Survey Best Practice
7
Survey Best Practice
Use in-class time
• Treat survey like an assignment
• Give proper time to complete
8
Survey Best Practice
•
Paper: In class surveys: ‘herd Instinct’
Most likely to complete
Dissatisfied
Satisfied
+30% higher response with
in-class paper surveys over
email Online surveys
9
Survey Best Practice
•
Online: In class surveys: ‘herd Instinct’
Most likely to complete
Satisfied
Dissatisfied
Same principle: higher response with in-class
Online surveys over email to participants
10
Survey Best Practice
We asked: Is it a Key Challenge to achieve
or maintain high response rates?
81% Strongly
agree
12% Nuetral
6% Disagree
11
Survey Best Practice
Why is obtaining high response rate important?
Institutional Research / Overall Student Satisfaction
Survey Universe
The University
Possible to draw some conclusions from a low response sample
12
Survey Best Practice
Why is obtaining high response rate important?
Course & Module Evaluation
Survey Universe
A 25% response rate
from a class of 20 is
only 5 students who’s
opinions may differ
greatly
The Course
The challenge is gaining representative data from each and every course or module
13
Survey Best Practice
Course & Module evaluation is not easy
• Its best in evaluating a course and the teaching to gain a
majority opinion 50%+ response
• It is important to be able to compare to show improvement
(against dept, university average, over time)
• Online survey systems fall short - average 15-30% response
rate
14
Survey Best Practice
Other considerations
• Gain stakeholder buy-in as the key to participate – The ‘myth of survey fatigue’
• Create surveys that make sense and report on only what you need to know
• Reinforce the concept of anonymity to get honest feedback
15
Efficiency and cost savings
Ways to drive efficiency and cost savings for surveys in
Higher Education
16
Efficiency and cost savings
#1 Centralise survey administration
• Avoid ‘academic run’ quality enhancement
• Run research projects on a common platform
• Know what surveys are being run at all times
17
Efficiency and cost savings
We asked: Who manages the survey process at
your organisation?
50%
65%
65% Centrally
47% Department or
curriculum area
47%
18
50% Individual
instructors
Efficiency and cost savings
We asked: Who has the main responsibility for
analysis and reporting at your organisation?
59% Centrally
15%
59%
38%
35%
35% Department or
curriculum area
38% Individual
instructors
15% Institutional
Research
19
Efficiency and cost savings
Academic run survey admin:
Individual Academic with 5 courses = 1 day per semester
20
Efficiency and cost savings
Academic run survey admin:
Department with 50 academics
21
Efficiency and cost savings
Arts
Law School
Business School
Languages
Engineering
Sciences
20 departments / 1000 instructors = 1000 teaching days per semester
22
Efficiency and cost savings
Main point:
• Funding cuts to teaching means less time and more work for
academics
• 1000+ teaching days per semester is better spent teaching
rather than on survey admin
23
Efficiency and cost savings
Centralised survey admin:
Best practice example: London School of Economics
23 Departments /1000+ staff / 9000 students
Complete survey administration 15 days per semester
(Deployment: 4 days / Data Capture: 10 days / Reporting: 1 day)
24
Efficiency and cost savings
#2 Create standardised questionnaires
• Develop compulsory question sets for lectures and modules
• Focus on teaching & learning and the student experience
Biology
Teaching
Quality & the
Student
Experience
English
25
Efficiency and cost savings
#3 Standardise core reporting functions
• Develop and implement a simplified reporting strategy
• Supply stakeholders only with what they need
• Standardised reporting allows for comparison to show trending over time
26
Efficiency and cost savings
#4 Utilise advanced technology
End of the day:
Surveys can be a powerful tool for making decisions and transforming an
organisation
• If you are invested in a survey system that produces low response rates
it is not going to meet the diverse requirements of Higher Education
institutions
• Survey administration should be a second job not a day job
27
Why Surveys?
Quality Assurance (Audit Culture)
• Inhibits learning from
outcomes
QAA
•
Quality
Assurance
Promotes
(Audit) incremental
improvement of academic
practice
Quality
Quality Enhancement (Feedback Culture)
• Encourages and requires frank
reporting
EESC OFFICER
• Facilitates transformational
change to surface & deep
conditions for learning
Enhancement
• Discourages risk taking
(low risk appetite)
• Supports and manages risk
taking
External Examination Sub Committee Officer
“(There is a need for) a more holistic view of how a University’s administration
and procedures affect a department’s teaching.”
Sue Wright, Enhancing the Quality of Teaching in Universities
28
Quality Assurance vs. Enhancement
EvaSys is designed to provide a ‘holistic’
platform to drive quality enhancement
efforts across the organisation
facilitating continuous improvement
Develop
Process
Plan for
Outcomes
Implement
Process
Evaluate
Process
29
Conclusion
Student Surveys - Best Practice
Eric Bohms
Presented by: Philip Wyse
Questions?
Thank you for your time!!
EvaSys Best Practice Blog: www.ericbohms.wordpress.com