On-line Learning Environment for
Multilevel Modelling
Fiona Steele and Sacha Brostoff
Centre for Multilevel Modelling
University of Bristol
The LEMMA Project

A node of the ESRC-funded National Centre for
Research Methods

LEMMA – Learning Environment for Multilevel
Methodology and Applications
–

2
www.cmm.bris.ac.uk
Research and training components
LEMMA Training

Capacity building in the analysis of data with complex
structure
–

Different modes of delivery
–
–
–
–
3
Ultimate goal is to move learners to “take-off”, i.e. conducting and
publishing multilevel analyses
Face-to-face workshops (3-day + 5-day with time allocated to
analysis of participants’ data)
Face-to-face workshop followed by on-line mentoring
Training for established networks (e.g. university departments)
Web-based materials in a virtual learning environment
Lessons Learnt

Participants need to be motivated and have time to learn
–
4
Best motivated are those with data and research questions that
can be addressed through MLM

Experience of on-line follow-up and targeting established
groups disappointing

Participants often do not possess pre-requisites for MLM (good
understanding of multiple regression)

In practical sessions, tendency to focus on mechanics of using
software rather than interpretation
Basic Principles

Accessible to anyone with basic statistics training (up to simple
regression)

Modules to have 2 integrated components: concepts and practice

Facility for learner’s self-evaluation
–


5
Pre-requisite quiz, and regular quizzes throughout materials
Collect data to evaluate materials and inform future training initiatives
–
Basic user profile information collected on registration
–
Quiz responses, webstats on patterns of use
Design materials so they can be easily modified by other trainers
Types of Material
For each module:
6

5-minute video giving overview of content

Prerequisites with links to other online resources

2 linked documents: (i) Concepts and methods, and (ii)
practice (MLwiN instructions with interpretation of output)

Quiz questions

Further reading (published research and other online
resources)

Glossary
Structure of Linked Documents

Concepts
–
–
–
–
–

Practice
–
–
–
7
30-40 pages, split into lessons
Illustrative examples from mix of disciplines
Draw links between fitted model equations, graphs of
predictions and verbal interpretation
No reference to software
Expect other trainers to use with little change
Each Concepts lesson followed by exercises in MLwiN
Thorough analysis and interpretation of one dataset
Trainers can rewrite for other datasets and software
Core Materials
1.
2.
3.
4.
5.
6.
7.
8
Types of variable
Introduction to statistical modelling
Multiple regression (single-level)
Data structures
Multilevel modelling of continuous data
Logistic regression (single-level)
Multilevel logistic regression
Future Materials

Substantive examples linked to core
materials

Models for other types of outcome
–

9
Nominal, ordinal, counts, duration
Models for non-hierarchical structures
The LEMMA VLE


Moodle
Open source, widely used
–

Has a good licensing model
–
–
–

10
Open University is involved in it’s development
Avoids annual fees
No restrictions on how many users
Sustainable
Does everything we need
Demonstrating the prototype

Encouraging engagement
–
–
–
–
11
Video overviews
Quizzes
Straw polls
ML Driver’s Licence
Further work






12
Improving navigation
Pilot testing
Link in to the follow up to Athens
Link administration with MLwiN UK free
version
More question types
Certainty based marking