Video usage in MOOCs – an explorative study
Carmen Zahn, Magdalena Mateescu, Jonas Kiener & Alessia Ruf
University of Applied Sciences and Arts, Northwestern Switzerland
Research Question:
How do specific elements of videos influence
the use of MOOCs and learning outcome?
Theoretical Background:
Rationale: Control over video information flow & sequencing
= positive influences on human information processing
Video functions influence learning outcomes
in procedural learning (Schwan & Riempp, 2014)
Video usage (especially rewind functions)
influences learning outcomes in hypervideo-based learning
(Zahn, Barquero & Schwan, 2004)
Video length influences engagement in MOOCs
(Guo, Kim & Rubin, 2014)
Agenda
◮
Overview over the MOOC, the sample and subsample
Research Question and Hypotheses
◮ Results
◮ Conclusion
◮
Agenda
◮
Overview over the MOOC, the sample and subsample
Research Question and Hypotheses
◮ Results
◮ Conclusion
◮
MOOC: Introduction to Work Psychology
MOOC Contents
Basic knowledge and central concepts of work psychology in 10
Chapters
•
e.g. paradigms of work organization and business
management; effects of work on humans; work
motivation and satisfaction; leadership, etc.
Learning goals: Basic understanding of psychological theories,
research questions and methods (specific for work and organizational
psychology); Knowledge about interrelations between work, health
and well-being; Application of the new knowledge for a deep
reflection of students`own life and work situation
MOOC-Structure
«x-MOOC» - 10 Chapters
Each chapter with 1 video, readings and quiz
«Meet the expert» online-session after Ch6
Participants :
2.299 persons enrolled
191 persons did not watch
any video
 Sample size: 2.108
Research Question:
How do specific elements of videos influence
the use of MOOCs and learning outcome?
Sample – Video Usage: Number of Persons per video
● 1.0 Einführung
2000
● 1.1 Gegenstand der wissenschaftlichen Psychologie
Anzahl Personen
1500
● 1.2 Aufgaben der (Angewandten) Arbeitspsychologie
● 1.3 Was bedeutet Arbeit im Leben v on Menschen?
● 1.4 Auf welchen Grundannahmen beruht die Arbeitspsychologie?
● 2.0 Einführung: Paradigmen der Arbeitsgestaltung und des Betr iebmanagements
1000
● 2.1 Ausdruck der Paradigmen der Arbeitsgestaltung, der Betr iebsorganisation und des Betr iebsmanagements
● 2.2 Paradigma des Taylorismus
● 2.3 Paradigma der Human Relations Be wegung
● 3.0 Einleitung: Job Enr ichment und Soziotechnischer Systemansatz
● 3.1 Herzberg und die Pittsb urgh−Studie / Job Enr ichment
● 3.2 der Soziotechnische Systemansatz der Arbeitsgestaltung, der Betr iebsorganisation und des Betr iebsmangements
● 3.3 Zusammenfassende Übersicht
● 4.0 Einleitung Arbeitstätigk eit und Arbeitshandlung
● 4.1 Tätigkeiten
● 4.2 die Arbeitshandlung
● 4.3 Struktur des Handelns, Teil 1
● 4.4 ●Struktur
des Handelns,
Teil 2von Arbeit 1
5.0 ●Einleitung
zur WirBeanspr
kung
5.1 ●Belastung,
uchung und
uchungsfolgen
5.2 Ermüdung,
Monotonie
und Beanspr
Sättigung
● 6.0 ●Einführung
dieStressoren
Wirkung der Arbeit 2
6.1 Stressinund
● 6.2 ●die
Ressourcen
6.3
und
Engagement
●Burnout
7.0 ●Einleitung
7.1
die Arbeitszufr iedenheit
und Arbeitsmotiv ation
●Arbeitszufriedenheit
7.2 in
und Motivation
●Arbeitszufriedenheit
7.3 ●Motivation
und
7.4 ●Motivationsprozess
(Vroom)
8.0 ●Einleitung
zu Gr
uppenarbeit und Arbeitsgr uppen
8.1Inhalt
●Gruppenarbeit, Arbeitsgr uppen
500
8.2 Typen von Gruppenarbeit und Partizipation
● 9.0 ●Einleitung
zu Management
und und
FührDefinition
ung
9.1 ●Management
... Herund
kunft
9.2 ●Führungsstile
Führ
ungsaufgaben und Führ ungsinstrum
9.3 Transaktionaler
Path−Goal−Ansatz
● 5.3 der Gesundheitsbegr iff
0
10
20
Videonummer
● 10.0 Einleitung zu psychologischen K onzepten und Ver
● 10.1 Die psychologische Arbeitsanalyse
● 10.2 Die MTO−Analyse
● 10.3 Die personenbez ogene Analyse un
30
40
Subsample: Persons who watched at least 70 % of Videos
An ah Pe onen
400
200
%
Mittelwert der angeschauten Videos
M
Mittel
600
S
0
0
10
20
Ange hau e V deo
30
40
Subsample characteristics
80
30
Mittelwert
60
count
count
20
40
10
20
0
0
20
40
60
Alter
80
Männlich
Weiblich
Geschlecht
Subsample: Country
Teilstichprobe 70% angeschauete Videos
SK − Slovakia
NO − Norway
MT − Malta
LI − Liechtenstein
LB − Lebanon
ID − Indonesia
HU − Hungary
CN − China
IT − Italy
GR − Greece
FR − France
ES − Spain
AT − Austria
CH − Switzerland
DE − Germany
0
25
50
75
count
10
0
12
5
Subsample Pre-Test: Intention to watch the lectures
Alle Teilnehmer
Teilstichprobe 70% angeschauete Videos
150
60
count
count
100
50
40
20
0
0
alle
die meisten
ungefähr die Hälfte
ein paar
m2
keine
alle
die meisten
ungefähr die Hälfteein paar
m2
Agenda
◮
Overview over the MOOC, the sample and subsample
Research Question and Hypotheses
◮ Results
◮ Conclusion
◮
Research Question:
How do specific elements of videos influence
the use of MOOCs and learning outcome?
Specific Elements of MOOC-Videos investigated
Folien
Variables:
Predictor:
⁃ Video length (min),
⁃ Number of embedded slides,
⁃ tips (yes/no)
Dependent Variables
⁃ Video Activity: Relative (per minute)
use of video functions: play, pause,
forwards, rewind.
⁃
MOOC-Engagement:
Video activity plus
number of chapters worked on
Quiz participated in
Number of trials in quizzes

Outcomes: Number of correct answers
to quiz questions per video
and per chapter relative to total number
of questions posed
General Hypotheses
◮
H1: Video features influence video activity
H2: Video features influence on MOOC-engagement.
◮ H3: Video features influence on outcome.
◮
Predictor Variables:
Predictor:
⁃ Video length (min),
⁃ Number of embedded Slides,
⁃ Tips (yes/no)
Dependent Variables
⁃ Video Activity: Relative (per minute)
use of video functions: play, pause,
forwards, rewind.
⁃
MOOC-Engagement:
Video activity plus
number of chapters worked on
Quiz participated in
Number of trials in quizzes

Outcomes: Number of correct answers
to quiz questions per video
and per chapter relative to total number
of questions posed
Video length
● c10_v2
20
● c8_v3
● c10_v4
● c5_v3
● c3_v3
● c7_v3
● c8_v2
● c6_v4
15
● c9_v4
● c7_v5
● c10_v3
● c7_v2
● c1_v5
● c4_v2
Videodauer (min)
● c9_v2
● c1_v2
● c3_v2
● c4_v3
● c2_v3
● c4_v4
● c6_v3
● c7_v4
● c1_v3
● c5_v2
● c9_v3
10
● c2_v2
● c5_v4
● c2_v4
● c6_v2
● c4_v5
● c5_v1
● c3_v4
5
● c1_v1
● c1_v4
● c4_v1
● c6_v1
● c10_v1
● c7_v1
● c8_v1
● c2_v1
0
● c9_v1
● c3_v1
10
20
Videonummer
30
40
Number of Slides
15
15
● c10_v2
● c10_v3
10
10
● c3_v3
● c5_v3
● c8_v2
● c9_v4
● c10_v2
● c10_v3
● c9_v3
● c1_v3
5
● c1_v2
● c2_v3
● c2_v4
● c3_v2
● c2_v2
● c6_v4
● c4_v3
● c4_v4● c5_v2
● c4_v5
● c1_v5
● c4_v2
● c1_v4
● c8_v3
● c7_v2
● c5_v4● c6_v3
● c1_v5
● c1_v1
0
● c2_v1
● c3_v1
● c4_v1
● c8_v3
● c7_v3
● c5_v1● c5_v4
● c6_v1
● c7_v1
20
Videonummer
30
● c7_v4
● c9_v2
● c8_v1● c9_v1
● c4_v5
● c10_v1
40
● c9_v2
● c9_v3
● c2_v2
● c6_v2
● c6_v3
● c9_v4
● c3_v4
● c5_v1
● c10_v4
● c7_v2● c7_v5
● c3_v2
● c3_v4
10
● c8_v2
● c6_v4
● c4_v3
● c1_v3
● c1_v1● c1_v4
● c2_v1
0
● c5_v2
● c2_v4
● c1_v2
● c10_v4
● c7_v4
● c7_v5
● c4_v2
● c3_v3
5
● c7_v3
● c6_v2
● c4_v4
● c2_v3
Anzahl Slides
Anzahl Fragen/Video
● c5_v3
● c3_v1
● c8_v1● c9_v1
● c4_v1
● c6_v1
● c10_v1
● c7_v1
0
0
10
20
Videonummer
30
40
Dependent Variables:
Predictor:
⁃ Video length (min),
⁃ Number of embedded Slides,
⁃ Tips (yes/no)
Dependent Variables
⁃ Video Activity: Relative (per minute)
use of video functions: play, pause,
forwards, rewind.
⁃
MOOC-Engagement:
Video activity plus
number of chapters worked on
Quiz participated in
Number of trials in quizzes

Outcomes: Number of correct answers
to quiz questions per video
and per chapter relative to total number
of questions posed
Data transformations
◮
Box-Cox Power Transformations to Multinormality: lambda =
(0.24 1.27 -0.75)
◮ All variables after z-transformation
Mitt
Mittelwert
N
w
MOOC
m
Method
Mixed model with fixed and random effects
◮
R nlme Packet, siehe Pinheiro J, Bates D,
DebRoy S, Sarkar D and R Core Team (2015). nlme: Linear
and Nonlinear Mixed Effects Models. R package version
3.1-122, http://CRAN.R-project.org/package=nlme
◮
For details, please contact: [email protected]
Agenda
◮
Overview over the MOOC, the sample and subsample
Research Question and Hypotheses
◮ Results
◮ Conclusion
◮
Results
◮
H1: Video features influence video activity
H2: Video features influence MOOC-engagement.
◮ H3: Video features influence outcome.
◮
H1: Video activity
◮
H1a: Video length influences video activity in negative way
(longer videos  lower activity; shorter videos  higher activity)
◮
H1b: Video-tips influence video activity in a positive way
(yes > no)
◮
H1c: Number of slides Video features influences video activity
(unspecified)
H1: Video features influence video activity
◮ Findings
Aktive Nutzung: standardisierte ß−Koeffizienten
Intercept
●
Videodauer (z−skalier t)
●
Tipp
●
Slides (z−sklalier t)
Videodauer*Tipp
• H1a: The shorter the videos
the higher the activity: ß = 0.13, t = -9.84, p < 0.001
• H1b: Video-tips (ja) influence
activity in positive ways: ß =
0.22, t = 11.54, p < 0.001
●
●
Videodauer*Slides
●
●
Tipp*Slides
●
• H1c: No influence of slide
number
●
Videodauer*Tipp*Slides
●
−0.2
−0.1
0.0
0.1
0.2
Bars denote CIs.
●
●
• Significant interaction
between video length, number
of slides and tips
H1: Video activity
--
Negative influence of video length
moderated by tips
Results
◮
H1: Video features influence video activity
H2: Video features influence MOOC-engagement.
◮ H3: Video features influence outcome.
◮
H2: MOOC-Engagement
◮
H2a: Video length influences MOOC-Engagement
(Shorter videos, more engagement)
◮
H2b: Video-tips (y) vs. no tips(n) influence MOOC-Engagement
◮
H2c: Number of embedded slides influence MOOC-Engagement
H2: MOOC-Engagement
◮ Findings
MOOC−Engagement: standardisierte ß−Koeffizienten
Intercept
●
Videodauer (z−skalier t)
●
Tipp
●
Slides (z−sklalier t)
●
Videodauer*Slides
●
●
Videodauer*Tipp*Slides
−0.3
• H2b: Tips (y) influence
engagement in a negative way:
ß = -0.16, t = -8.42, p< 0.001
●
Videodauer*Tipp
Tipp*Slides
• H2a: The shorter the videos the
more engagement: ß=-0.17, t=12.77, p <0.001
●
−0.2
−0.1
0.0
0.1
0.2
0.3
0.4
• H2c: Fewer slides  more
engagement: ß = -0.04, t = 3.36, p< 0.001
Bars denote CIs.
●
• Significant interactions between
video length, number of slides
and tip.
H2: MOOC-Engagement
-
negative influence of video length
stronger for videos without tips
•
-
Influence of number of slides
moderated by tips
Results
◮
H1: Video features influence video activity
H2: Video features influence MOOC-engagement.
◮ H3: Video features influence outcome.
◮
H3: Outcome (Quiz)
◮
H3a: Video length influences outcomes
(shorter videos  better outcomes in quizzes)
◮
H3b: Video-tips (y) vs. no tips(n) influence outcome
(tips (y) improve outcome).
◮
H3c: Number of slides influence outcome
H3: Outcome
◮ Findings
• H3a: Shorter videos  better
Faktisches Lernergebnis: standardisierte ß−Koeffizienten
outcomes in quiz: ß=-0.19,
t=-10.90, p <0.001
• H3b: Tips  higher outcome:
ß=0.18,t=-8.76, p <0.001
• H3c: Smaller number of slides
influence outcome in positive
way: ß=-0.03, t=-2.13, p <0.05
Intercept
●
Engagement (z−skalier t)
●
Videodauer (z−skalier t)
●
Tipp
●
Slides (z−sklalier t)
●
Engagement*Videodauer
●
Engagement*Tipp
●
Videodauer*Tipp
●
Engagement*Slides
●
Videodauer*Slides
●
Tipp*Slides
●
Engagement*Videodauer*Tipp
●
Engagement*Videodauer*Slides
●
Engagement*Tipp*Slides
●
Videodauer*Tipp*Slides
Engagement*Videodauer*Tipp*Slides
−0.4
●
●
−0.2
0.0
0.2
Bars denote CIs.
0.4
0.6
• Significant Interactions
between video length,
number of slides and tips,
e.g., high ß for interaction
video length and tips:
ß=0.40,t=13.09, p <0.001
H3: Outcome
Interaktion: Videodauer * Tipp
Faktisches Lernergebnis (z−scores)
2
1
tip
kein Tip
Tip
0
−1
−2
−2
−1
0
Videodauer (z−scores)
1
2
Agenda
◮
Overview over the MOOC, the sample and subsample
Research Question and Hypotheses
◮ Results
◮ Conclusion
◮
Tentative Conclusion:
Hypothesis
IV/DV
a
Video Length
b
Video tips
c
Number of Slides
H1: Video
activity
Shorter videos,
more active use
If yes,
more activity
H1c -
H2: MOOCEngagement
Shorter videos
more engagement
If yes,
less engagement
Fewer slides,
more engagement
H3: Outcome
Shorter videos
better outcome
In quizzes
If yes,
better outcome,
Few slides ,
Beter outcomes
 Especially for
videos without tips
 For longer videos
 Especially for
Videos with tips ?
Implications
Outcome (Quiz):
⁃ shorter videos (related research)
⁃ tips for longer videos
⁃ not to many slides
Bedienelemente: play, pause,
forward, rewind, ...
tip
MOOC-Engagement:
⁃ short videos with few embedded slides
⁃ not necessarily provide tips
Video mit
Folien
tip
Gesamtstichprobe: Alter, Geschlecht
50
Mittelwert
40
200
30
20
100
10
0
0
20
40
Alter
60
80
Weiblich
Männlich
Geschlecht
Anderes
Gesamtstichprobe: Abschluss
Ja, ich bin Vollzeitstudent/in bzw. Schüler/in.
Ja, ich bin Teilzeitstudent/in.
Nein, ich befinde mich in keinem Ausbildungsprogramm.
count
Gesamtstichprobe: In welchen Land wohnst du jetzt?
VU − Vanuatu
SK − Slovakia
RU − Russia
RS − Serbia
PL − Poland
PE − Peru
NO − Norway
NL − Netherlands
MT − Malta
LI − Liechtenstein
LB − Lebanon
KZ − Kazakhstan
ID − Indonesia
GB − United Kingdom
EG − Egypt
EC − Ecuador
CA − Canada
BR − Brazil
BG − Bulgaria
BE − Belgium
AL − Albania
MA − Morocco
HU − Hungary
CN − China
IT − Italy
GR − Greece
FR − France
ES − Spain
AT − Austria
CH − Switzerland
DE − Germany
count