Ongoing Process of Learning Outcomes Assessment

Recognition of Lifelong
Learning Model
@
HBMSU
Dr. Mansoor Al Awar
Chancellor
International Conference on ICT and Post – 2015 Education
Leveraging ICT, Transforming Education
23-25 May 2015 , Qingdao China
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
1
Content
The presentation provides an overview of the following:
I.
Learning philosophy at HBMSU
II.
HBMSU Lifelong Learning Model
III. Academic Excellence at HBMSU
IV. Assessment of Learning Outcomes at HBMSU
V. Ongoing Process of Learning Outcomes Assessment: Casual Learners
VI. Ongoing Process of Learning Outcomes Assessment: Committed Learners
VII. Ongoing Process of Learning Outcomes Assessment: Concentrated Learners
VIII. Ongoing Process of Learning Outcomes Assessment: Continuing Learners
2
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Opening Message
“I am convinced that teaching our children in the same way
we did yesterday is in itself perpetuating a new kind of
illiteracy that no longer has a place in any society that wishes
to join that global race.”
H.H. Sheikh Mohammed bin Rashid Al Maktoum,
UAE Vice President, Prime Minster, Ruler of Dubai
My Vision: Challenges in the Race for Excellence, page 196, English version
3
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Learning philosophy at HBMSU
• The HBMSU philosophy is based on vital methods that have been developed
through extensive research and development by specialist from all over the
world.
• The methods include:
o Offering high quality programs
o Supporting the pursuit of lifelong learning
o Promoting the freedom of information
Accessibility
Flexibility
Affordability
4
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Prospective fruits of HBMSU Learning
philosophy
Affordability
Flexibility
Accessibility
Virtuality
Diversity
Feasibility
More Educated
Community
5
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
HBMSU Learners’ Categories:
4Cs’ Model Lifelong Learning Model (LLL)
Outcomes
Offerings
Customized & Solution-Driven
Programs
Continuing Learners
Executive Development
Research-Driven
Concentrated Learners
Innovative Academic Degrees
(Bachelors, MSc. PhD)
Committed Learners
Professional Certification & Diploma
(< 9 months)
Casual Learners
Open Access Education (Smart
Campus, Short Courses, Seminars,
and Conferences
Competency-Based
Programs
Informal Learning
6
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Create new
learning
pathways and
arrangements
Developed based
on labor market
intelligence and
feedback from
industry and key
stakeholders from
UAE and the
region
Continuously
monitored and
reviewed for
improvement
Characteristics
of HBMSU
Lifelong
Learning Model
Opportunity to
progress from
one level to a
subsequent
level
Opportunity for
international
mobility of
learners
Possible
opportunity for
recognition of
Prior Learning
(according to
NQA model)
7
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Academic Excellence at HBMSU
o Licenced from the Ministry of Higher Education and Scientific Research since
2009.
o 22 academic programs accredited from the Commission for Academic
Accreditation (CAA) at the Ministry of Higher Education and Scientific Research in
UAE.
o Integrated Quality Management System: CAA standard; International Organization
for Standardization (ISO) standards: ISO 9001: 2008, ISO/IEC 27001:2005, and
ISO 14001:2004; and British Standards Institute (BSI): OHSAS 18001:2007.
o Middle East e-Learning Quality Framework (MeLQ): A multi-dimensional framework
for quality in technology-enhanced learning which developed as joint research
project between HBMSU and the Swiss Centre for Innovations in Learning (scil).
8
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Development
- Resources
availability
- Cost
- Effectiveness/ROIs
- Risk Management
Organizational
Dimensions
- Faculty & Staff
- Information to
prospect learners’
- Intuitional
effectiveness
- Responsiveness to
needs
- Learners’ Services
Technological
Dimensions
- Target Population
Learning & Teaching
Dimension
- Stakeholders’
perception of change
- Workload,
compensation matters
- Ownership and IP
- Philosophy of
change
& Strategy
- Teaching & Learning
Approaches
- Learning Outcomes
- Learners’ Interaction
- Learners’
Assessment &
Feedback
- Instructional design
- Program/ Course
Review
Economical
Dimension
Strategic Dimension
- Institutional Mission
Change Dimension
MeLQ Framework
- E-Learning Tools
and Infrastructures
- IT Strategies
- Accessibility &
Usability
- Content Reusability
- Information security
& data discovery
9
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Assessment of Learning Outcomes at HBMSU
o
HBMSU adopts an ongoing multi-dimensional assessment and improvement process for each
of the four categories of learners: Casual, Committed, concentrated, and continuing.
o
HBMSU prepares a comprehensive plan for the assessment and improvement process which is
intended to assist administration, faculty, staff, and learners in HBMSU on the direct and
indirect assessment methods used to demonstrate the attainment of the Intended Learning
Outcomes (ILOs), Program Outcomes (POs) and the overall performance, the process for
utilizing data from these assessments to modify and improve learning and institutional
effectiveness.
o
The assessment and improvement process covers different levels of assessment:
1.
Learners’ assessment,
2.
Course/ unit level assessment,
3.
Curriculum level assessment,
4.
Program level assessment,
5.
Institutional level assessment.
10
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Assessment of Learning Outcomes
at HBMSU
3
Assessment
and
Improvement
Process
Set a target or
criteria for
success (expected
results)
4
Carry out the
assessment and
collect evidence
(assessment
results).
2
1
Define
Learning
Outcome
Choose a method to measure the
outcome (assessment method):
•
Multiple assessment measures
or Triangulation,
•
Direct and indirect,
•
Quantitative and qualitative,
•
Reliable and valid
5
Discuss the
evidence & use
results for
improvement:
Changes to Course or Unit
• Pedagogical Practices;
Prerequisites; Content
Changes to the Program
• Program Outcomes;
Requirements;
Courses/Units; Duration;
Admission Criteria
Changes to Academic
Processes
• Frequency/ Schedule of
Offerings; Technology;
Instructors Training;
Advising Standards or
Processes
Changes to Assessment Plan
• ILO Statement(s);
Assessment Methods;
Data Collection Methods;
Analysis Processes;
Stakeholders; Frequency.
11
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
HBMSU Lifelong Learning Model:
Ongoing Process of Learning Outcomes Assessment
Continuing
Learners
Concentrated
Learners
Committed
Learners
Casual
Learners
Casual
Learners
12
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Ongoing Process of
Learning Outcomes
Assessment
Casual Learners:
Open Access Education (Cloud Campus, Short
courses, Seminars, Conferences)
Open Access
Education Overall
Assessment
Frequency: Every time the unit is
offered (applicable to Cloud Campus
and Short Courses)
Course/Unit
Level
Assessment
Frequency: Semiannual
(applicable to Cloud Campus and
Short Courses)
Frequency: Once a year
Interactive Technology-enhanced Formative
Assessments:
- Unit-embedded questions (web-based
clickers,..),
- Online quizzes and tests (with
embedded visuals)
- Game-like learners response tools
- Hypothetical situation responses,
- Psychometric Tests
Learners’ performance on each
ILO based on course/ unit
assessment
Stakeholders Surveys (Participants,
government representatives, private
sector representatives, ..)
Learner
Assessment
Interactive Technology-enhanced
Summative Assessments (for Certification
Purposes):
- Online Unit Exam,
- Interactive slideshows,
- Expert evaluation of practical skills
using rubric,
- Technology-enhanced Portfolio
assignments,
- National or International Licensure
Exams (if any)
Support Team Survey
ILOs Surveys
Learners Satisfaction Survey
National Test, Local Examinations
and Tests, Licensure Exams (if
applicable)
Support Team (unit / course level)
Survey
National Qualifications Authority
(NQA) reviews (if applicable)
Focus group meeting (Support
Team, learners, administrators)
Management review meeting
(Support Team, Content providers,
administrators)
13
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
HBMSU Lifelong Learning Model:
Ongoing Process of Learning Outcomes Assessment
Continuing
Learners
Concentrated
Learners
Committed
Committed
Learners
Learners
Casual Learners
14
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Ongoing Process of
Learning Outcomes
Assessment
Committed Learners:
Professional Certificates and Diplomas (< 9 months)
Learner
Assessment
Course Level
Assessment
Program/ Diploma
Level Assessment
Frequency: Every time the
course is offered
Frequency: Every time the
course is offered
Frequency: Every 2 years
Case studies, hypothetical situation
responses
Learners’ performance on each
ILO based on course assessment
NQA reviews
Capstone or Graduation Project
National Test, Local Examinations
and Tests, Licensure Exams (if any)
administrators)
Practical Assignments
Course-embedded questions
Portfolio assignments
ILOs Surveys
Employers Surveys
Performance appraisal of in-class
exercises
Line Manager Evaluation
Assessment of projects with standard
rubric
Learners Satisfaction Survey
Peer or expert evaluation of practical skills
using rubric
Employer Surveys
External Examiner’s Report
Focus group meeting
(Instructors, Trainers, learners,
administrators)
Program/ Diploma review meeting
(Instructors, Trainers, learners,
Psychometric Tests
Licensure Exams
Graduate Surveys
15
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
HBMSU Lifelong Learning Model:
Ongoing Process of Learning Outcomes Assessment
Continuing
Learners
Concentrated
Concentrated
Learners
Learners
Committed
Learners
Casual Learners
16
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
University
Council
Management Review
Meetings
Ongoing Process
of Learning
Outcomes
Assessment
Concentrated
Learners:
Innovative
Academic
Degrees
(Bachelors,
MSc. PhD)
School and
Academic
Councils
Assessment
Report
Program Level
Assessment
Graduates Exit
Survey
Advisory board
survey
Alumni survey
Comprehensive
Instructor
Course Review
Report
Focus
Groups
Course Files
Curriculum
Level
Assessment
Employer survey
CAA’s report
Outcome
assessment
ILOs Survey
(Faculty)
Course
Level
Assessment
ILOs Survey
(Learners)
Faculty Members
17
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Course Level Assessment
Objectives
•
•
•
•
Responsibilities
Faculty  Focus group Department Chair  School Board
Learners Assessment
Tools
I. Direct Assessment Methods
A. Interactive Curriculum / Course-Related Assessment
Methods
•
Course-embedded questions
•
Case studies, hypothetical situation responses
•
Assessment of projects with standard rubric
•
Research papers
•
Performance appraisal of learners participation and
involvement (discussion boards, blogs,…)
Course Assessment
tools
•
•
•
•
•
•
Frequency
Every semester the course is offered
Review course objectives and ILOs
Review the level of achievements of the course ILOs
Establish effective communication between the faculty and course focus group
Suggest and implement improvements
B. Examinations and Tests
•
Test-embedded questions
•
Standardized Examinations
and Tests
•
Pre-post test
•
Licensure Exams (In any)
II. Indirect Assessment Methods
•
ILOs Survey (Learners)
•
ILOs Survey (Faculty)
Learners’ performance on each ILO based on course assessment
ILOs Survey (Learners)
ILOs Survey (Faculty)
Learner Satisfaction Surveys
Comprehensive Instructor Course Review Report
Course focus group meetings
18
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Curriculum Level Assessment
Objectives
•
•
•
•
Ensure match between Program Outcomes (POs) and
courses’ ILOs.
Ensure that courses serve their outcomes in the curriculum.
Establish effective communication between faculty.
Suggest and implement improvements
Responsibilities
Focus groups Department Chair School Council
Assessment
tools
•
•
•
•
Frequency
Once a year
Graduates Exit Survey
Focus group meetings
Curriculum review meetings (annually)
Faculty retreat (Fall semester)
19
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Program Level Assessment
Objectives
•
•
•
•
•
•
•
Develop POs that meets the School and University missions.
Monitor overall POs and graduate attributes.
Devise clear advising, monitoring, and assessment plans.
Implement well structured curriculum.
Identify assessment tools for continuous monitoring and improvement of the
program.
Collect and analyze assessment data.
Devise action plans for continuous improvement.
Responsibilities
Faculty members School Council Academic Council University Council
Assessment
tools
•
•
•
•
•
Frequency
Minor review every year
Major review every 3 years
Program reviews
CAA reviews
Employers, Alumni, and Graduate Exit Surveys
External Examiner’s Report
Advisory Board meetings
20
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
HBMSU Lifelong Learning Model:
Ongoing Process of Learning Outcomes Assessment
Continuing
Continuing Learners
Learners
Concentrated
Learners
Committed
Learners
Casual Learners
21
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
Ongoing Process of
Learning Outcomes Continuing Learners:
Executive Development Programs
Assessment
Learner
Assessment
Course Level
Assessment
Frequency: Every
time the course/ unit
is offered
Frequency: Once a
year
Interactive unitembedded questions
Technology-enhanced
hypothetical situation
responses
Technology-enhanced
portfolio assignments
Online Standardized
Tests
Learners’ performance on
each ILO based on course/
unit assessment
National and
International
Level
Assessment
Impact on
Organization
and Society
Related International
Awards (FLEA, Kano
Medal,…etc.)
Impact on learner’s
professional
development
Intended Learning
Outcomes (ILOs) Survey
Learners Satisfaction
Survey
Support Team Survey
Focus group review
meeting (Support Team,
learners, administrators)
NQA reviews (if
applicable)
National Test, Local
Examinations and Tests,
Licensure Exams (if
any)
Impact on learner’s
organization
Impact on local,
regional, and global
society
22
Copyright © 2015 Dr. Mansoor Al Awar – All Rights Reserved
谢谢
Thank
You
[email protected]
@DrMansoorAlAwar
alawarmansoor
Mansoor Al Awar
23