FREMA: e-Learning Framework
Reference Model for Assessment
David Millard
Yvonne Howard
Learning Technology Group
University of Southampton, UK
Structure
• Overview
– Introduction to FREMA
– What is a Reference Model?
– Views on the Model
• Do we have the right Personas – are you there?
• Do our Concept Maps make sense to you?
• Workshop: Eliciting Common Usage Patterns
– What are the most common tasks within assessment?
– What is (and is not) being addressed by current software?
– What do you consider to be the core elements of the
assessment domain?
Introduction
• What is FREMA?
– JISC funded Project between Southampton, Strathclyde and Hull
– Part of the e-Learning Framework (ELF) effort
• What is the ELF?
– Service Oriented Architecture for e-learning systems
– Layered Web Services (Domain services over Common Services)
– Dynamic and evolving
• FREMA will develop a Reference Model for the Assessment
domain
– Supporting design-time activities
– Supporting run-time activities
– Virtual organisations and lifelong learning
• What is a Reference Model?
– A description of how services behave within a particular domain
– A community resource
Assessment and ELF
•
•
•
•
Not enough to describe and define these services
Need a proper audit trail of decision making
Start by defining the domain
Work up through the services to a reference
implementation
• This is an ELF Reference Model
Anatomy of a Reference Model
• Domain Definition
– Overview of the domain, and how projects and standards fit within it
• Identifying Common Usage Patterns
– Scoping the FREMA Project
• Developing Use Cases
– Formal descriptions of usage patterns
• Gap Analysis
– Mapping of Use Cases to the Services in ELF
Reference
Impl’
• Service Profiles
Service Profiles
– Formal descriptions of those services
• Reference Implementation
–
–
–
–
Of key/core services
Examples
Validation
Resource
Gap Analysis
Use Cases
Use Cases
Use Cases
Common Usage Patterns
Assessment Domain Definition
What does it look like?
• An evolving, cross-referenced, searchable web site
•
•
•
•
Indexed resources and narrative descriptions of the domain
UML Use Cases and Scenario documents
Service descriptions, narrative and WSDL
Service implementations to download (Java/.NET)
• Different gateways into the model
according to how you want to use it
Reference
Impl’
Service Profiles
Gap Analysis
Use Cases
Use Cases
Use Cases
Common Usage Patterns
Assessment Domain Definition
How might you use it?
• Use the Reference Implementation
– Build on some or all of the developed services
• Use the Service Profiles
– To develop your own services that will fit into the framework
• Use the Use Cases
– To help understand usage patterns within the domain
– Develop new Service Profiles and thus Services
Reference
Impl’
• Use the Domain Definition
–
–
–
–
To develop a context for your own work
Understand how existing work fits together
Identify standards
Locate experts
Service Profiles
Gap Analysis
Use Cases
Use Cases
Use Cases
Common Usage Patterns
Assessment Domain Definition
Use Case for a Service Oriented
Reference Model (SORM)
A More Concrete Description
• An ontology for the resources of the domain
• A knowledge base of the resources
• Concept Maps that define the domain and aid searching
– Processes (think of verbs)
– Classes of Objects (things, think of nouns)
– And many more gateways into resources
• Visualisations of
– Concepts
– Populations of resources over concepts
• Searchable, flexible, dynamic website
Process
concept
map
(verbs)
Classes
concept
map
(nouns)
Another look at the SORM Use Case
How would these actors choose to view the domain?
Personifying the actors
•
Will, Web services developer
–
Scenario:
‘I want to lookup use cases and scenarios to help me design my application. This will
help me to define my footprint in the assessment domain. I see there are some web
services I could download but some are missing. What standards can I use when
writing my own web services to ensure that I can interoperate with the web services
I’ve chosen?’
•
Yvonne, Institutional Resource Manager
–
Scenario
‘I want an overview of what this domain is all about. I want to know what standards
are applicable in the domain to ensure that we comply with quality assurance
requirements. I want to examine use cases and scenarios to understand the
available footprints. I also want to know who the key players are and what the key
projects are.’
•
Other Actors
–
–
–
–
Early Adopter
Toolkit developer
Course developer
And there are others . . .
Concept modelling
• What mental models might these and
other actors use to orient their searches?
• How do they visualise the assessment
domain?
Structure
• Overview
– Introduction to FREMA
– What is a Reference Model?
– Discussion
• Do we have the right Personas?
• Do our Concept Maps make sense?
• Workshop: Eliciting Common Usage Patterns
– What are the most common tasks within assessment?
– What is (and is not) being addressed by current software?
– What do you consider to be the core elements of the
assessment domain?
Another concept map – based on a domain expert
view
Tracking
Artefacts
Student Appeal
Evidence
Learning
Authorise
Student Feedback
Group
Peer Group
Authenticate
Multiplicity
Course
Group
Structuring
Schedule
Assessment
Self assess
Mark
Delivery
Items
Validation
Authoring
Plagiarism
Workflow
Final Grade
Transcript
QA
Failover
Marking
Grade book
Member
Help