Developing an Evaluation Plan for MIT OpenCourseWare April 9, 2003

April 22, 2004
University of Notre Dame
A New Model for Open Sharing
Anne H. Margulies and Jon Paul Potts
Agenda
I.
Vision
II. Implementation
III. Impact
2
Vision
Institutional Decision-Making
• Fall 1999 — Faculty committee appointed
• Fall 2000 — “OpenCourseWare” concept recommended to MIT
President Charles M. Vest
• April 2001 — MIT OCW announced in The New York Times
3
Vision
Institutional Decision-Making
“OpenCourseWare looks counterintuitive
in a market-driven world. But it really is
consistent with what I believe is the best
about MIT. It is innovative. It expresses our
belief in the way education can be
advanced – by constantly widening access
to information and by inspiring others to
participate.”
– Charles M. Vest,
President of MIT
4
Vision
Vision to Reality
• June 2001 — Funding partnership with the William and Flora Hewlett
Foundation, and the Andrew W. Mellon Foundation
• September 2002 — MIT OCW Pilot site opened to the public
– 50 courses from 23 academic disciplines
• September 2003 — MIT OCW officially launched
– 500 courses from all five MIT schools and 33 academic disciplines
• April 2004 — 200 additional courses, bringing total to 701
5
Vision
What Is MIT OCW?
MIT OpenCourseWare IS NOT: • An MIT education
• Intended to represent or replace the
interactive classroom environment
• A distance education initiative
MIT OpenCourseWare IS: • A Web-based publication of virtually all
MIT course content
• Open and available to the world
• A permanent MIT activity
6
Vision
Why Is MIT Doing This?
• Furthers MIT’s fundamental mission
• Embraces faculty values
– Teaching
– Sharing best practices with the greater community
– Contributing to their discipline
• Counters the privatization of knowledge and champions the movement
toward greater openness
7
Vision
Dual Mission
• Provide free access to MIT course materials for educators and learners
• Create a model other universities may use to publish their own course
materials
8
Vision
Where We Are
701 Courses
Phase I
Pilot
Courses
Phase III
Steady State
Phase II
Expansion
2002
2003
2004
2005
2006
2007
2008
50
500
900
1250
1550
1800
1800
Publication • Design pub process
• Implement technology
strategy
• Develop IP strategy
• Implement dept.
liaison program
Evaluation • Develop evaluation
strategy
• Conduct baseline
evaluation
Outreach • Partner with Universia
(translation affiliate)
•
•
•
•
Inventory content and improve quality
Enhance site features and functions
Add video materials
Plot new content capture tactics
Each year:
• Add new courses: ~100
• Revise existing: ~ 275
• Archive old:
~ 100
• Implement reporting strategy
• Conduct annual evaluations and focused studies
• Conduct annual
evaluations and studies
•
•
•
•
• Collaborate with
consortium members
Facilitate other opencoursewares
Partner with translation/distribution affiliates
Build awareness
Foster learning communities
9
Implementation
10
Implementation
Publishing 500 Courses
Site Highlights
4Syllabus
4Course Calendar
4Lecture Notes
4Assignments
4Exams
4Problem/Solution Sets
4Labs and Projects
4Simulations
4Tools and Tutorials
4Video Lectures
11
Implementation
Depth and Breadth
12
Implementation
Publication Process
Managing a Course Through the MIT OCW Process
Plan
Recruit
faculty
and
courses
• Transcribe,
convert
materials
• Identify IP
• Design layout
Publish
Build
• Input content
•
•
•
•
Add metadata
Scrub content
Clear IP
Initial QA
•
•
•
•
Test site
Final QA
Faculty signoff
Stage for
publish
Support
• Edit/add
• Respond to
inquiries
• Troubleshoot
MIT OCW =
Snapshot of
Completed
Course
13
Implementation
Technology
MIT Facilities
Publishing
Environment
Origin Server
Search, Feedback
Content Distribution Network (Akamai)
Thousands of servers around the world
deliver MIT OCW course materials
14
Impact
15
Impact
Access Data
MIT OCW Weekly Visits, October 1, 2003 to April 17, 2004
16
Where We Are
Access Data
Site Traffic Overview
Since
10/1/03*
December
January
February
March
Page Views
20,604,427
2,680,794
3,311,611
2,884,061
3,025,412
Daily Visits
*11,103
9,276
11,624
11,174
10,891
Monthly Visits
*301,719
287,546
360,360
324,058
337,620
First-Time
Visits
*174,407
172,536
196,710
174,961
187,348
Monthly Repeat
Visits
*127,312
115,010
163,650
149,097
150,272
* Figures in italics are averages
17
Impact
Access Data
March 2004
Country
Hits
Country
Hits
1 India
954,167
11 Brazil
340,281
2 Canada
859,782
12 France
334,190
3 China
822,206
13 Spain
318,292
4 United Kingdom
672,339
14 Indonesia
251,495
5 South Korea
448,975
15 Australia
240,689
6 Japan
421,334
16 Turkey
239,972
7 Germany
402,965
17 Colombia
196,504
8 Vietnam
401,498
18 Singapore
185,495
9 Taiwan
392,701
19 Mexico
165,221
366,484
20 Greece
164,496
10 Italy
18
Impact
Access Data
• Self-learners are 52% of visitors
– 5774 daily visits
– 60% of North American visitors are self-learners
• Students are 31% of visitors
– 3442 daily visits
• Educators are 13% of visitors
– 1443 daily visits
– 55% of educators teach at 4-year colleges or the equivalent
– 49% have less than 5 years teaching experience
• Almost 70% of users have a bachelors degree or higher
19
Impact
Use Data
Use Scenario
% of Use
Planning, developing or teaching a course
36%
Enhancing personal knowledge
22%
Planning curriculum
10%
Other
32%
Complementing a subject currently taking
43%
Enhancing personal knowledge
40%
Planning future course of study
10%
Other
7%
Enhancing personal knowledge
81%
Learning subject matter—course not available for
study
9%
Planning future course of study
8%
Other
5.7% response rate on 21,500 surveys
2%
20
Impact
Impact Data
• 92% of visitors satisfied with quality of the course materials
• 95% said they would return to the MIT OCW Web site for future use
• 99% said MIT OCW will have an “extremely positive” or “moderately
positive” impact on education around the globe (83% “extremely
positive”)
• Over 47% of educators have reused MIT OCW materials (or plan to);
41% may reuse materials in the future
• 76% of educators agree that MIT OCW will impact their future teaching
practices
21
Impact
Feedback Data
• 15,000 emails to [email protected]
– Majority (60+ percent) are grateful or congratulatory
– Other inquiries
• How to register
• Technical questions
• Inquiries from other educators
• Vendors
– Negative responses (less than 3 percent)
• 21,000 users self-subscribed to monthly email newsletter
22
Impact
MIT Use Data
• Over half of MIT faculty have participated so far
• Most MIT faculty are satisfied with process and with helpfulness of
staff*
• Process not overly burdensome: 42% spent <5 hours preparing
materials for publication, 33% 5 – 10 hours*
• 32% of MIT faculty report using MIT
OCW to advise students, do
research, and (most often) prepare
to teach*
MIT.EDU WEEKLY VISITS
• Dramatic spike in internal MIT
traffic during February registration
period suggests MIT students use
MIT OCW as aid in course selection
* Source: MIT Faculty Survey, 13.5% response
on 950 surveys
23
Impact
Benefits for MIT
• Institute-level benefits
– Advances MIT’s institutional mission: “To advance knowledge and
educate students in science, technology, and other areas of
scholarship that will best serve the nation and the world in the 21st
century”
– Enhances MIT’s image around the world
– Generates community pride (alumni)
– Stimulates collaboration among faculty
• Department-level benefits
– Showcases individual departments and their curricula
– Enhances faculty and student recruitments efforts
– Accelerates adoption of the Web
24
Impact
Emerging “opencoursewares”
• Other OCWs are beginning to
appear
• Some using MIT materials, some
using the format, some using the
idea
25
Impact
Translations
• 50 courses in Spanish
and Portuguese site
through Universia.net
partnership
• Individual courses in 10
languages
26
Impact
Recognition
MIT faculty’s vision, and MIT OCW implementation have been recognized.
January 29, 2003
The Kyoto (Japan)
Digital Archives
Project
recognizes
MIT OCW
for:
Vision
Content
November 10, 2003
MIT OCW/ Sapient
partnership
recognized with
“InfoWorld 100”
award for
• MIT OCW
Technology
October 15, 2003
Sapient receives
“Microsoft Internet
Business
Solution
of the Year” C E R T I F I E D
award for:
………………………
• MIT OCW
Business Solutions
Technology
Partner
December 22, 2003
MIT OCW/ Sapient
partnership
recognized by
Computerworld
Honors Program
for:
• Vision
• Technology
October 21, 2003
Massachusetts
Interactive
Media Council
honors MIT
OCW for:
• Design
• User
Experience
April 20, 2004
MIT OCW
recognized by
the Webby
Awards for:
• Vision
• User
Experience
27
Impact
What Does It Mean?
• Continues to be tremendous excitement
• The vision is achievable
• The impact of MIT OCW will be significant
28
Impact
What Is An “opencourseware”?
• Course materials created by faculty (and sometimes other colleagues or
students) to support teaching and learning
• Offers materials free-of-charge and is universally accessible via the Web
• Materials represent a substantially complete set of materials used in the
course (minimum of syllabus, course calendar, and lecture notes or
equivalent)
• Is intellectual property-cleared
• Permits use, reuse, adaptation (derivative works), and redistribution of
the materials by others
29
Thank You!
Visit MIT OpenCourseWare online at
http://ocw.mit.edu
Visit the “Opencourseware How To”
site on the Web at
http://ocw.mit.edu/OcwWeb/HowTo/index.htm
30