TRANSFORMING
LEARNING ENVIRONMENTS
THROUGH COURSE REDESIGN
TODAY’S DISCUSSION



The National Center for Academic
Transformation
Overview of the Methodology and Findings
of the Program in Course Redesign
Proven Models for Successful Redesign
•
•
•
Established in 1999 as a university
Center at RPI funded by the Pew
Charitable Trusts
Became an independent non-profit
organization in 2003
Mission: help colleges and universities
learn how to use technology to
improve student learning outcomes
and reduce their instructional costs
BRIEF HISTORY OF
COURSE REDESIGN
•
•
•
•
•
The Roadmap to Redesign (R2R)
2003 – 2006 (20 institutions)
Colleagues Committed to
Redesign (C2R)
2006 - 2009 (60 institutions)
Programs with Systems & States
2006 – present (~80 institutions)
The Redesign Alliance
2006 – present (70+ institutions)
Changing the Equation
2009 – 2012 (25+ institutions)
TRADITIONAL INSTRUCTION
Seminars
Lectures
“BOLT-ON” INSTRUCTION
WHAT’S WRONG
WITH THE LECTURE?
•
•
•
•
•
Treats all students
as if they are the
same
Ineffective in
engaging students
Inadequate
individual
assistance
Poor attendance and
success rates
Students fail to
retain learning
WHAT’S WRONG WITH
MULTIPLE SECTIONS?
•
•
•
•
•
In theory: greater interaction
In practice: large class size
In practice: dominated by the
same presentation
techniques
Lack of coordination
Inconsistent outcomes
PROGRAM IN
COURSE REDESIGN
To encourage colleges
and universities to
redesign their
approaches to
instruction using
technology to achieve
cost savings as well as
quality enhancements.
30 projects
50,000+
students
WHAT DOES NCAT MEAN BY
COURSE REDESIGN?
•
•
•
Course redesign is the process of
redesigning whole courses (rather than
individual classes or sections) to achieve
better learning outcomes at a lower cost by
taking advantage of the capabilities of
information technology.
Course redesign is not just about putting
courses online.
It is about rethinking the way we deliver
instruction in light of the possibilities that
new technology offers.
WHY REDESIGN?
Look for courses where redesign will
have a high impact:
•
•
•
•
•
•
•
High withdrawal/failure rates
Students on waiting lists
Students turned away – graduation bottleneck
Over enrollment of courses leading to
multiple majors
Inconsistency of preparation
Difficulty getting qualified adjuncts
Difficulty in subsequent courses
QUANTITATIVE (13)
•
Mathematics
–
–
–
–
–
–
–
Iowa State University
Northern Arizona
University
Rio Salado College
Riverside CC
University of
Alabama
University of Idaho
Virginia Tech
•
Statistics
–
–
–
–
•
Carnegie Mellon
University
Ohio State University
Penn State
U of Illinois-Urbana
Champaign
Computer
Programming
–
–
Drexel University
University at Buffalo
SCIENCE (5)
SOCIAL SCIENCE (6)
•
Biology
–
–
•
–
University of Iowa
U of WisconsinMadison
Astronomy
–
U of ColoradoBoulder
Psychology
–
Fairfield University
University of
Massachusetts
–
–
Chemistry
–
•
•
–
•
Sociology
–
•
Cal Poly Pomona
University of Dayton
University of New
Mexico
U of Southern Maine
IUPUI
American
Government
–
U of Central Florida
HUMANITIES (6)
•
English Composition
–
–
•
Spanish
–
–
•
Portland State University
University of Tennessee
Fine Arts
–
•
Brigham Young University
Tallahassee CC
Florida Gulf Coast University
World Literature
–
University of Southern Mississippi
TEAM EFFORT IS KEY
Each team included
–
–
–
–
Administrator
Faculty experts
Technology expertise
Assessment assistance
IT IS POSSIBLE TO INCREASE
LEARNING WHILE REDUCING COST
•
•
•
25 of 30 PCR projects
improved learning; the
other 5 showed equal
learning.
24 measured course
completion rates; 18
showed improvement.
All 30 reduced costs by
37% on average, with a
range of 15% to 77%.
Program in Course Redesign
WHAT HAPPENS TO THE
SAVINGS?
•
•
•
•
•
•
Stay in department for continuous course
improvement and/or redesign of others
Provide a greater range of offerings at upper division
or graduate level
Accommodate greater numbers of students with same
resources
Stay in department to reduce teaching load and
provide more time for research
Redesign similar courses
Miscellaneous
– Offer distance sections
– Reduce rental expenditures
– Improve training of part-time faculty
NCAT METHODOLOGY:
Relevance and Utility
•
•
•
•
•
•
Discipline: math &
literature
Age: traditional & working
adults
Institution: small & large
Location: on-campus & at
a distance
Redesign: current & new
courses
Level: introductory &
advanced
WHAT DO THE FACULTY SAY?
•
•
•
“It’s the best experience
I’ve ever had in a
classroom.”
“The quality of my worklife
has changed
immeasurably for the
better.”
“It’s a lot of work during
the transition--but it’s
worth it.”
SIX REDESIGN MODELS
•
Supplemental
•
Replacement
•
Emporium
•
Fully Online
•
Buffet
•
Linked Workshop
Add to the current structure
and/or change the content
Blend face-to-face with
online activities
Move all classes to a lab
setting
Conduct all (most) learning
activities online
Mix and match according to
student preferences
Replace developmental
courses with just-in-time
workshops
REDESIGN CHARACTERISTICS
•
•
•
•
•
•
•
Redesign the whole course—not just a
single class
Emphasize active learning—greater
student engagement with the material
and with one another
Rely heavily on readily available
interactive software—used
independently and in teams
Mastery learning—not self-paced
Increase on-demand, individualized
assistance
Automate only those course
components that can benefit from
automation—e.g., homework, quizzes,
exams
Replace single mode instruction with
differentiated personnel strategies
Technology enables good pedagogy with large #s of students.
SUPPLEMENTAL MODEL
•
•
Retains the basic structure, especially class
meetings
Supplements lectures and textbooks with
technology-based, out-of-class activities to
–
–
•
encourage greater student engagement with course
content
ensure that students are prepared when they come to
class
May also change what goes on in class by
creating an active learning environment
within a large lecture hall setting.
BIOLOGY
University of Massachusetts
•
•
•
•
CHALLENGES
Inconsistent student preparation
Poor class attendance
Lectures that repeated the contents of the
textbook
High dissatisfaction with course by both
faculty and students
BIOLOGY
University of Massachusetts
•
•
•
•
•
•
Continue to have large class meetings
Require short pre-tests before the start of the
first class each week and these are available
for the entire term as review
Receive small number of points for taking the
online quiz
Provide 24/7 online study materials
Include small group interactions during class
focused on applied biology problems
Class periods are now used to discuss
biology problems, rather than lecture
BIOLOGY
University of Massachusetts
Student Outcomes
•
•
•
In spite of more difficult questions, scores on
exams in the redesigned course averaged
73% vs. 61% in the traditional course.
23% of the exam questions in the traditional
model required reasoning or problem solving
skills vs. 67% in the redesigned course.
Attendance averaged 89.9% in the redesigned
course vs. 67% in the traditional course.
REPLACEMENT MODEL
•
•
•
•
•
Reduces the number of in-class
meetings
Replaces in-class time with online,
interactive learning activities
Determines what activities require
face-to-face and what can be done
online
Provides 24/7 access to online learning
resources
Includes online self-assessment
activities with immediate feedback
COMPUTER LITERACY
Arizona State University

GOALS
Update content and delivery




Cover modern methods and technologies
Promote self-directed learning
Use modern learning environment and technology to
deliver course
Reduce cost

Decrease delivery costs – efficient sustainable delivery
structure for ~2200 students each term
COMPUTER LITERACY
Arizona State University
Updated Content
Traditional Course


4 paper-based, multiple choice exams based on computer
technology too introductory for today's tech-savvy students
12 assignments in Microsoft Word, Excel, and FileMaker Pro,
submitted in hard copy
Redesigned Course



9 online quizzes demonstrating understanding of modern
computing concepts
7 self-guided learning assignments applying computing concepts
and computer-driven problem solving techniques, submitted via
various online methods
4 major projects requiring substantial independent inquiry,
submitted via various online methods
COMPUTER LITERACY
Arizona State University
Updated Delivery

Traditional Model





2 lectures per week
Open lab hours for assistance, staffed by TAs and graders
Assignments turned in hard copy during lecture – graded by
graders, and returned hard copy during lecture
Paper-based, multiple choice exams completed in lecture
Redesign Model




1 optional lecture per week, all content on Blackboard
Scheduled guidance in lab, staffed by Undergraduate Learning
Assistants (ULAs)
Discussion board available for assistance daily
All assignments, quizzes and projects submitted via the Web
COMPUTER LITERACY
Arizona State University
Learning Outcomes:

Traditional Sections


Average of 26% of students earned 70% or more
Redesigned Sections

Average of 65% of students earned 70% or more, on a
demonstrably more difficult course
Cost Reduction: a reduction from $50 to $35
per student, which is a 30% savings.
EMPORIUM MODEL
•
•
•
•
•
Eliminates all lectures
Replaces them with a learning-resource
center (lab) model
– interactive software
– on-demand, personalized assistance
Permits the use of multiple kinds of
personnel
Allows multiple courses to be offered at
the same time and place
Can be adapted for different kinds of
institutions and disciplines
THE EMPORIUM MODEL
77% Cost Reduction (V1)
30% Cost Reduction (V2)
THE MATH EMPORIUM
at Virginia Tech
•
•
•
•
Traditional
38 sections (~40)
10 tenured faculty,
13 instructors, 15
GTAs
2 hours per week
$91 cost-per-student
•
•
•
•
Redesign
1 section (~1520)
1 instructor, grad &
undergrad TAs + 2
tech support staff
24*7 in open lab
$21 cost-per-student
Replicated at U of Alabama, U of Idaho, LSU, Wayne
State, U Missouri-St. Louis, Seton Hall, Cleveland
State CC, Northeast State CC, Jackson State CC
UNIVERSITY OF ALABAMA
Math Learning Center
UNIVERSITY OF ALABAMA
Success Rates
Semester Success Rate
Semester
Fall 1998
Fall 1999
Fall 2000
Fall 2001
Fall 2002
Fall 2003
Fall 2004
Fall 2005
Fall 2006
Fall 2007
Fall 2008
Spring 1999
Spring 2000
Spring 2001
Spring 2002
Spring 2003
Spring 2004
Spring 2005
Spring 2006
Spring 2007
Spring 2008
47.1%
40.6%
50.2%
60.5%
63.0%
78.9%
76.2%
66.7%
73.8%
75.2%
78.1%
Success Rate
44.2%
53.5%
35.8%
49.8%
41.8%
55.4%
60.1%
56.6%
59.8%
57.3%
FULLY ONLINE MODEL
•
•
Eliminates all in-class meetings
and moves all (or most) learning
experiences online
Adopts successful design
elements of other models
including
–
–
–
–
web-based, multi-media resources
commercial software
automatically evaluated assessments
with guided feedback
links to additional resources and
alternative staffing models
FINE ARTS
Florida Gulf Coast University
CHALLENGES
•
•
•
•
Significant inconsistency among multiple
sections
Difficulty finding either faculty or adjuncts
with the breadth of knowledge in all of the
humanities
Poor performance in this course that is
required by all freshmen
Growth in students and no money for new
faculty
FINE ARTS
Florida Gulf Coast University
•
•
•
•
•
Each module covers one aspect of the
Humanities
Each module is designed and monitored by a
faculty expert in that academic area
One course coordinator manages the course
of 400+ students each term
Undergraduate peer tutors and adjuncts
guide discussion groups and evaluate longer
papers
24/7 interactive learning resources are
available anytime, any place
FINE ARTS
Florida Gulf Coast University
•
•
•
•
Traditional
25 sections (~30); 6
sections (~15) = 800
Taught mainly by
adjuncts
“Course drift”
$132 cost-per-student
•
•
•
•
Redesign
Single section (~950)
Taught by 1 faculty, 1
course coordinator, 20
preceptors
Consistent & coherent
$81 cost-per-student
Average exam scores increased from 70% to 85%
Number of A’s/B’s increased from 31% to 75%
DFW rate decreased from 45% to 11%
EMERGENT LITERACY
Arizona State University
•
•
•
•
•
Traditional: Graduate level course taught f2f with
FT faculty once a year on multiple campuses in
sections of ~30 – enrollment ~100 now, but cannot
meet demand
Redesign: Students from any campus enroll
online; course taught every term with enrollment
projected to grow to 300-500 annually; GTAs lead
online course with FT faculty oversight
Learning assessed through common finals and use
of common rubrics for written work
No difference in learning or change in DFW rates
Plan is to change all 14 courses in the MS degree
BUFFET MODEL
•
•
•
•
•
Assess each student’s knowledge/skill level
and preferred learning style
Provide an array of high-quality, interactive
learning materials and activities
Develop individualized study plans
Built in continuous assessment to provide
instantaneous feedback
Offer appropriate, varied
human interaction when
needed
STATISTICS
Ohio State University
CHALLENGES
•
•
•
•
•
•
•
Previous redesign using IT increased the cost
Students had highly variable learning styles
Lectures were poorly attended
20% of the students repeat the course each
quarter even though most have satisfactorily
completed initial modules
Too many emails for faculty
Faculty time was used inefficiently
Inconsistency among sections
STATISTICS
Ohio State University
•
•
•
•
•
Students use online assessment by Felder
and Solomon.
There are multiple routes to established
outcomes for each module.
Students are assisted in thinking about how
they approach learning and what mode is
easiest for them.
Students file a learning plan for each module.
Various kinds of learning activities using
websites, software, video lectures, small
group discussions, individual and group
projects.
STATISTICS
Ohio State University
OUTCOMES
•
•
•
Redesign students had greater success on
common exams (mean = 78.3) than
traditional students (mean = 70).
The number of students needing to retake
the course was reduced from 33% to 12%.
Cost reduction from $191 per student in the
traditional to $132 per student in the
redesign
LINKED WORKSHOP MODEL
•
•
Retains the basic structure of college-level course,
especially class meetings
Replace remedial/developmental course with just-intime (JIT) workshops
–
–
–
–
•
designed to remove deficiencies
computer-based instruction, small-group activities and test
reviews
individually assigned modules based on diagnostic
assessments
facilitated by students who have previously excelled in core
course who are trained and supervised by core faculty
JIT workshop activities designed so students use
concepts during next core course class session, which
in turn motivates them to do workshop activities
DEVELOPMENTAL MATH
Austin Peay State University
Student Success Rates
College Course
Before
SLA
Fund of Math
32.4%
69.9%
Elem Statistics
22.4%
52.5%*
* Higher than the success rate for students with 19-22 ACT subscores
A STREAMLINED REDESIGN
METHODOLOGY
“A Menu of Redesign Options”
•
•
•
•
•
•
•
•
Six Models for Course
Redesign
Five Principles of Successful
Course Redesign
Cost Reduction Strategies
Course Planning Tool
Course Structure Form
Five Models for Assessing
Student Learning
Five Critical Implementation
Issues
Planning Checklist
MANY DIFFERENT COURSES
•
Mathematics
–
–
–
–
–
–
–
–
–
Developmental Math
Pre-calculus Math
College Algebra
Discrete Math
Introductory Algebra
Elementary Algebra
Beginning Algebra
Intermediate Algebra
Linear Algebra
•
Statistics
–
–
–
–
•
Business Statistics
Introductory Statistics
Elementary Statistics
Economic Statistics
Computing
–
–
–
–
–
Computer Programming
Information Technology
Concepts
Computer Literacy
Information Literacy
Tools for the Information
Age
•
SCIENCE
–
–
–
–
–
–
Anatomy and
Physiology
Astronomy
Biology
Ethnobotany
Chemistry
Geology
•
SOCIAL SCIENCE
–
–
–
–
–
American
Government
Macro and
Microeconomics
Psychology
Sociology
Urban Affairs
HUMANITIES
•
•
•
•
•
•
•
•
•
•
•
•
•
•
British Literature
Communication Studies
Developmental Reading
Developmental Writing
English Composition
European and US History
Great Ideas in Western Music
History of Western Civilization
Public Speaking
Spanish
Technical Writing
Visual & Performing Arts
Women & Gender Studies
World Literature
PROFESSIONAL
•
•
•
•
•
•
•
Accounting
Education: The Curriculum
Elementary Education
Engineering
Nursing
Nutrition
Organizational Behavior
FACULTY BENEFITS
•
•
•
•
•
•
Increased opportunity to work directly with
students who need help
Reduced grading
Technology does the tracking and monitoring
More practice and interaction for students without
faculty effort
Ability to try different approaches to meet different
student needs
Opportunity for continuous improvement of
materials and approaches
TRANSFORMING
LEARNING ENVIRONMENTS
THROUGH COURSE REDESIGN
Carolyn Jarmon, Ph.D.
[email protected]
www.theNCAT.org
BREAK
WHAT IS THE MOST IMPORTANT
THING THAT WE HAVE LEARNED
ABOUT QUALITY AND COST?
The factors that lead
to increased student
learning and
increased student
retention are the
same as those that
lead to reduced
instructional costs!
#1: REDESIGN THE WHOLE
COURSE SEQUENCE
–
Quality: Eliminate “course drift”;
greater course coherence and quality
control
–
Cost: Eliminate duplicate effort;
create opportunities for alternate
staffing
#2: ENCOURAGE ACTIVE LEARNING
–
Quality: “Learning is not a
spectator sport.”
–
Cost: Reduce faculty preparation
and presentation time; reduce
grading time (e.g., interactive
software)
#3: PROVIDE STUDENTS WITH
INDIVIDUALIZED ASSISTANCE
–
Quality: Students get help when they
are “stuck” and stay on task rather
than giving up (e.g., software
tutorials, F2F in labs)
–
Cost: Apply the right level of human
intervention (e.g., tutors, course
assistants)
#4: BUILD IN ONGOING ASSESSMENT
AND PROMPT (AUTOMATED) FEEDBACK
–
Quality: Enables practice, diagnostic
feedback, focused time on task
–
Cost: Good pedagogy with large
numbers of students; automated
grading; faculty spend time on what
students don’t understand
#5: ENSURE SUFFICIENT TIME ON
TASK AND MONITOR STUDENT
PROGRESS
–
Quality: Self-pacing vs. milestones for
completion; points for engagement
–
Cost: Course management systems
can reduce costs while increasing
oversight; automated intervention;
focus on students who need help
FIVE PRINCIPLES OF
SUCCESSFUL COURSE REDESIGN
#1: Redesign the whole course
#2: Encourage active learning
#3: Provide students with
individualized assistance
#4: Build in ongoing assessment
and prompt (automated) feedback
#5: Ensure sufficient time on task
and monitor student progress
FIVE CRITICAL
IMPLEMENTATION ISSUES
•
•
•
•
•
Prepare students (and their parents) and
the campus for changes in the course.
Train instructors, GTAs and
undergraduate peer tutors.
Ensure an adequate technological
infrastructure to support the redesign as
planned.
Achieve initial and ongoing faculty
consensus about the redesign.
Avoid backsliding by building ongoing
institutional commitment to the redesign.
AFTER LUNCH
ASSESSMENT GOAL
To establish the
degree to which
improved
learning has been
achieved as a
result of the
course redesign.
ASSESSMENT PLANNING
Step 1. Establish the
method of obtaining data.
Step 2. Choose the
measurement method.
ESTABLISH THE METHOD
OF OBTAINING DATA
•
•
Baseline “Before”
(traditional) and
“After” (redesign)
Parallel Sections –
Compare traditional
sections and
redesigned sections
CHOOSE THE MEASUREMENT
METHOD: FIVE MODELS
A. Comparisons of Final Exams
B. Comparisons of Common Content
Items Selected from Exams
C. Comparisons of Pre- and Post- Tests
D. Comparisons of Student Work using
Common Rubrics
E. Comparisons of Course Grades using
Common Criteria
COST SAVINGS GOAL
Create cost savings
that can be used to
sustain ongoing
redesign, to fund
future operations and
to free up resources
for program and/or
institutional priorities.
WHAT’S YOUR ENROLLMENT
SITUATION?
•
•
Is your enrollment
growing or
projected to grow?
Is your enrollment
stable or
declining?
ACCOMMODATE
ENROLLMENT GROWTH
•
•
•
Increase the number of sections, keep
section size the same and keep
personnel the same.
Reduce the number of sections,
increase the section size and change
the mix of personnel.
Change the mix of personnel teaching
the course.
Mix and match for greater savings!
U OF TENNESSEE
Spanish
•
•
•
•
•
Traditional
57 sections (~27)
Adjuncts + 6 TAs
100% in class
$167,074
($2931/section)
1529 students @ $109
•
•
•
•
•
Redesign
38 sections (~54)
Instructor-TA pairs
50% in class, 50% online
$56,838 ($1496/section)
2052 students @ $28
STABLE COURSE ENROLLMENT
•
•
•
Reduce the number of sections and
increase the section size. (Reduce the
number teaching the course.)
Reduce the number of graduate
teaching assistants (Only 9 of 30
projects!)
Change the mix of personnel teaching
the course (Adjuncts, undergraduate
learning assistants.)
Mix and match for greater savings!
VIRGINIA TECH
Linear Algebra
•
•
•
•
Traditional
38 sections (~40)
10 tenured faculty,
13 instructors,
15 GTAs
2 hours per week
$91 cost-per-student
•
•
•
•
Redesign
Single section (1520)
1 tenured faculty,
graduate & undergraduate assistants
24 x 7 in open
computer lab
$21 cost-per-student
FAIRFIELD UNIVERSITY
General Biology
•
•
•
•
•
Traditional
7 sections (~35)
7 faculty
100% wet labs
$131,610
$506 cost-per-student
•
•
•
•
•
Redesign
2 sections (~140)
4 faculty
50% wet, 50% virtual
$98,033
$350 cost-per-student
INNOVATIVE COURSE
REDESIGN PRACTICES
•
•
•
•
•
•
Creating "Small" within
"Large“
Undergraduate Learning
Assistants
Freshmen Don’t Do
Optional
Modularization
New Instructional Roles
Avoiding “Either/or”
Choices
REDESIGN CHECKLIST
•
•
•
•
•
•
Consensus about Curriculum
Course Organization
Materials Selection and Adaptation
Faculty Development and Training
Student Preparation
Infrastructure
ASSIGNMENT
•
•
•
•
•
•
What are the obstacles to redesigning your
course at UNC Charlotte?
What issues do you need to consider?
What evidence would help you overcome
the obstacles?
What information do you need to gather?
What process, if any, might help overcome
the obstacles?
Choose one person to make note of any
specific questions.