Links in the Chain: turning learning analytics data into actions
ABLE Project 2015-1-BE-EPPKA3-PI-FORWARD
STELA Project: 562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Development of the NTU Student Dashboard
Learning Analytics
"Analytics is a term used
in business and science
to refer to computational
support for capturing
digital data to help
inform decision-making
… Learning Analytics
appropriates this concept
for education.”
(Buckingham Shum,
2012, p.1)
(Clow, 2012)
Why NTU is interested in learning analytics?
• Retention
• Belonging
• Attainment
• Research & improving the student experience
• What works? 1
• Internal audit
• Entrepreneurial IS team
Developmental Cycle
Sept
2013
Sept
2014
Pilot Phase
Sept
2015
Feb
2016
Phase One
Phase
Two
8 of 9 Schools
Governance
Problem solving
Ethics
 All Schools
 New data
sources
 Assessment
view
 Personalisation
 Embedding into University
systems
Whole
University roll
out: increased
awareness
Increasing resources (e.g.
how to guides),
communication, guidance
 4 courses
 40 staff
 500 1st years




Willing participants:
very positive staff
feedback, limited
student awareness
Near to whole
University roll out:
increased awareness
Phase Three
Staff and student consultation ongoing throughout developments
Further details of projects at http://www.ableproject.eu/project-outputs/ and
https://eng.kuleuven.be/english/projects/STELAproject/stela
The NTU Student Dashboard
• Can be viewed as two products:
Physical
Dashboard
Algorithm
• Staff and students interact with physical dashboard
• Algorithm is the behind the scenes, learning analytics element
What does the Dashboard do?
Student biographical
info, e.g. enrolment
status
Can make
comments in
free text box
Staff
view
Student
view
Evidence of student
engagement
•
Door swipes
•
•
•
Library books
NOW use
Dropbox
submissions
•
•
Attendance data
Access to ebooks and
journals through
Shibboleth
authentication
(where appropriate)
NTU
Student
Dashboard
Compares student
engagement across
the cohort & gives
rating
Raises
alerts!!
Potential benefits of learning analytics for personal
tutors
Data accuracy for algorithm spotting students
at risk
• Two big questions:
1. Can the algorithm correctly identify at risk students?
2. Can it do so on a timescale that allows intervention?
Relationship between yearly average engagement &
progression
• Low average (mode) engagement for the year is an indicator of risk
Relationship between term one average engagement &
progression
• Low average (mode) engagement for the 1st term is an indicator of risk
No engagement alerts
• Any one alert is an indicator of risk
• Students with multiple alerts had lower incidence of progression
Class view
• Designed so staff have easy access to student data.
• Allows staff to quickly identify potentially at risk students
Links to student’s dashboard
Able to sort on headings
Individual student view
Not Fully
Enrolled
Staff and students can
benchmark engagement –
springboard for conversation
Notes and referrals
• Notes inputted by
staff only
• Time and date stamp
• Referral to:
 Library Academic
Skills
 Student Support
Services
 Employability
(planned)
Personal tutors can track interactions with students
Personal tutors can make referral whilst in room with student
Student profile
•
Basic information (ability to
report if this is wrong)
•
Engagement summary
•
Entry qualification details
•
Engagement history (for
previous years)
•
Details to help early tutorials
Attendance
•
Data drawn
automatically from
the attendance app
•
The wheel shows
overall attendance
•
The table below
provides more
detailed information
for recent weeks
•
Springboard for
conversation
(potential context of
School attendance
policy)
Assessment & Feedback View
•
Only show
assessments and
feedback
submitted
through NOW
(the University’s
VLE)
•
Shows
assessments and
feedback for
multiple modules
•
Better sight of student performance than only seeing own module results so
tutor can make more informed recommendations
The challenges of converting student information to
actionable intelligence
How are we using the Dashboard to address
disparities in attainment?
• Two change agents
Students
Staff
• Developing learning analytics is challenging
• Institutional change based on learning analytics may be a different
order of magnitude
– Institutions already have data on students at risk, attendance, non-submission
of coursework
– Yet it is often extremely difficult to change student outcomes
Exploring the NTU Student Dashboard
• Checking their engagement scores and increasing the amount of time they spend studying
were the top explorations of the Dashboard amongst students.
Checked your own engagement score
24%
Increased the amount of time you spend studying
36%
13%
36%
35%
5%
34%
19%
Compared your engagement score with other students on your
course
15%
24%
34%
28%
Changed your behaviour to raise or maintain your engagement
score (for example made sure that you swiped to go into a
building)
14%
26%
33%
28%
Spoke to your tutor as a result of looking at information on the
5%6% 13%
Dashboard
77%
Spoke to someone providing specialist help (for example student
support services/ library) as a result of looking at information on 4%5% 12%
the Dashboard
0%
Very Often
Often
Have you logged into the NTU student Dashboard?
When using the Dashboard, how often have you explored the following?
Base: 515 (2016), 469 (2015)
10%
20%
Sometimes
79%
30%
40%
Never
50%
60%
70%
80%
90% 100%
Relationship between number of Dashboard
log-ins and engagement classification
Some groups are more likely to log in more frequently, e.g. males, A level students
How are we engaging with Staff as Change
Agents?
• Changing personal tutoring policy
• Providing evidence through research
– Induction activity
– Student surveys and interviews
– iPad trial
• Making the systems better
– Notes and referrals function
• Staff communication
– Briefings and drop-in sessions
– Newsletter term 1 – low engagement
– Newsletter term 2 – alerts
Thank you for listening.
Any questions?