PLA: Portfolio Assessment
and the Kresge Pilot for
LearningCounts
Today’s Agenda
1. Overview
2. What it is and what it’s not!
3. The Process
4. Faculty/Evaluator Perspective
5. LearningCounts.org and Kresge Pilot
6. Wrap-up/Q&A
Meet Our Experts!
Dr. Chari Leader Kelley
Senior Fellow
Council for Adult and Experiential Learning
(CAEL)
Chari has worked to improve access to higher
education for adults and historically under-served
students. Chari uniquely understands the
challenges adults face in juggling the responsibilities
of work, family, and college through her personal
experience, her research endeavors and in working
at colleges that primarily serve adults – Bellevue
University and Excelsior College, in senior
leadership positions.
At CAEL, Chari devoted two years to building
LearningCounts.org and working with various
colleges, universities, systems, and states to
improve PLA opportunities for their students. Now a
Senior Fellow, Chari assists CAEL on various state
initiatives that lead to changes in policy and practice.
Meet Our Experts!
Dr. Melissa Nemon
Senior Researcher
Sillerman Center for the Advancement of Philanthropy
Brandeis University
Melissa holds a Bachelors in Psychology, a
Masters in Community Social Psychology, a
second Masters in Community Economic
Development with a specialization in Public
Policy, a Ph.D. in Community Economic
Development. In addition to her work at Brandeis
University, Melissa currently operates her own
consulting firm.
She has worked with local communities, state
and federal government agencies; international
communities; and a variety of nonprofit, for profit
and NGOs (non-governmental organizations)
from all over the world. She has been teaching
(face-to-face and online) since 2008.
Meet Our Experts!
Kristen Himmerick
Senior Director
CAEL’s
LearningCounts.org
Kristen joined CAEL in June 2012 as Director of Academic
Technology and Student Engagement of LearningCounts. Today,
she is the Senior Director of LearningCounts, leading all internal
LearningCounts operations.
Before joining CAEL, Kristen was part of the Innovation Lab at
Southern New Hampshire University, where she helped write the
proposal for a self-paced, competency based associate’s degree
that was awarded the Next Generation Learning Challenge Grant
for College Completion (SNHU’s College for America Program).
Kristen also worked for over 9 years at the City Colleges of
Chicago, where she managed academic technology at the District
Office. Kristen earned an M. Ed. from the University of Illinois and a
Bachelor’s of Science in Communications from Northwestern
University.
5
Overview
Portfolio assessment is the most difficult PLA
method for policy makers to understand and
agree upon.
Skepticism, myths, and bad practices color nearly
every portfolio assessment discussion –
 Our goal today is to demystify the portfolio
assessment process, provide best practices
and quality assurance information, as well as
address all of your questions.
PLA Methods
Standardized Exams
(like CLEP, DSST, UExcel, AP, IB)
Evaluated Non-College
Training and Education
Programs by ACE, NCCRS,
College Credit,
Retention
and
Completion
or Your Faculty
Faculty-Developed
Course Challenge
Exams
Industry recognized
Certificates & Licenses
Student Portfolios &
Demonstrations
(Individual
assessments)
How PLA Helps Graduation Rates
Associate
Degree
students are
two times as
likely to
persist to
graduation.
Portfolio Assessment Success
It is important to develop a portfolio
framework and protocols that all
institutions can adopt, to confidently award
credit with the portfolio assessment
method.
• Student-centric
• Ensure consistency
• Ensure academic integrity within, across
and between institutions
• Improve portability of credit awards
Key CAEL Standards
Credit is for
LEARNING, not
for experience
Subject matter
experts make
credit
recommendations
Any fees are for
assessment, not
for the amount of
credit awarded
What it isn’t…..
•
•
•
•
A stack of papers
Easy, quick, or a slam dunk
Done in isolation
Block credit or credit not related to
the program of study
• Less academically sound or
rigorous than taking the course
• A subjective judgment
What it is…..
For students: Earning credit via
portfolio assessment means
demonstrating your learning and
essentially PROVING it.
 Learning must align with learning outcomes and
competencies on the syllabus
 Documentation supports learning claims
 Narrative is thoughtful, reflective and thorough
 Rubric is known
 Minimum of “C” grade -- All or nothing
Aligning Learning Outcomes
 From the course syllabus and description
 May include demonstrable competencies,
particularly in technical, the arts, and
workforce specialties (performance assessment)
 Must show an understanding of knowledge
area combined with the application of it
 Must articulate what you know and can
demonstrate (specific)
 Integrative/reflective
 Documentation/evidence
 Assessed with a Rubric
Best Practices: The Process
Student meets with an academic advisor to
discuss PLA options.
 Review degree/certificate requirements
 Determine if the student believes s/he
knows information covered in a required
course or possible elective.
 Choose PLA method – Is portfolio
assessment the right way to go?
 If portfolio is the only available option, is
the student right for the portfolio method?
Advisor approves the decision and next
steps.
Best Practices
Student takes PLA course, workshop, or
seminar to learn how to properly prepare a
portfolio.
 Student reflects upon previous learning and
experiences, develops a timeline of formal
and informal learning experiences.
 Student reviews course syllabi, textbooks,
and other information regarding her/his
knowledge set.
 Student selects course(s) and double-checks
to make sure they are not duplicative. We
encourage the student to check again with
her/his academic advisor.
Student creates first draft (electronic).
Best Practices
Student uses the course description, learning
objectives, and outcomes to begin organizing
a narrative, documentation, and the “case for
credit.”
 Student begins a narrative (APA style) that
addresses each outcome with how it was learned,
any theoretical constructs, how it was applied and
can be applied today.
 Student makes requests for “letters of verification”
from supervisors and experts.
 Student finds actual work products that can be
used as evidence (presentations, publications,
videos, etc.).
Best Practices
Student uses the assessment rubric tool to
ensure all of the requirements for a successful
portfolio are met:
 Student must cite sources (including syllabus
and course description).
 Student must link evidence/documentation to
the learning outcomes (according to the
syllabus).
 Student must write succinctly and at the
college level – emphasizing the learning, not
the experience.
 Once the student has submitted the portfolio,
s/he cannot make further changes to it.
How It Works
Communication 2110 Example
Public Speaking Goals of the Course
•
Students should demonstrate basic oral communication skills necessary for
functioning effectively in the classroom and ultimately taking them into the
workplace for participation as a competent citizen.
•
Students should gain an appreciation for the oral communication process by
understanding the importance of the speaker-audience situation and learning
basic strategies for effectively communicating and overcoming potential
obstacles in the speaking
•
Students should develop critical thinking skills and active listening skills by
learning to listen to others and how they are most influenced.
•
Students will have the opportunity to maximize leadership skills that can be
practiced in formal speaking situations. Attention will also be given to speech
anxiety, organization, library research, persuasion, audience analysis, and
credibility.
•
Students will gain technology experience from learning to make a presentation
with PowerPoint.
Example
COMMUNICATION 2110 – PUBLIC SPEAKING
Learning Goal
Demonstration
Evidence
1. Demonstrate basic oral
Narrative:
communication skills
necessary for functioning
effectively in the classroom and
ultimately taking them into the
workplace for participation as a
competent citizen
Examples from personal
experience noting what was
learned and how it applies to the
Learning Goal.
Reference work situation.
Work Products:
• 2 Performance Evaluations
noting high marks in
interpersonal communications
• Supervisor comments on how
well the employee performs on
the team.
Narrative:
•
2. Gain an appreciation for the
oral communication process by
understanding the importance
of the speaker-audience
situation and learning basic
strategies for effectively
communication and
overcoming potential obstacles
in the speaking situation.
Cite academic sources.
3. Demonstrate critical thinking
and active listening skills by
listening to others and
determining how they are most
influenced.
Narrative:
Write about attending a Hillary
Clinton speech and rally. Point to
lessons learned by watching the
audience and the press.
Write about how
speaker/audience connection,
lessons learned, and an example
of an obstacle overcome when
chairing the company United Way
campaign.
Video of United Way
Presentation (Persuasive) and
student’s introduction of the CEO
•
Letter of verification from
supervisor
•
Outline and notes from
preparation for the
presentation.
•
Video of Rotary Club speech
when student used a Clinton
technique in persuading the
audience to help get out the
vote. (Persuasive and Informative)
Example
COMMUNICATION 2110 – PUBLIC SPEAKING
Learning Goal
Demonstration
Evidence
4. Opportunity to maximize
leadership skills that can be
practiced in formal speaking
situations – also address
speech anxiety, organization,
library research, persuasion,
audience analysis and
credibility.
Narrative:
Address all topics – Use
citations. Describe learning how
to address anxiety by being well
organized, prepared, and
practicing, as well as the
student’s experience with a
speech coach.
Citations
5. Demonstrate technology
experience by making a
presentation with PowerPoint
Narrative:
Discuss how many times the
student has used PowerPoint at
work and how he learned to
improve them.
Two PowerPoint presentations
done during the last six months
Topics from Syllabus:
Letter of Verification from Speech
Coach
Recording of student’s webinar for
church (informative speech)
Narrative:
All presentation/speech examples




Introduction Speech
Informative Speech
Demonstration Speech
Persuasive Speech
Write about each speech type,
cite sources. Point to the various
types of speeches/presentations
in the documentation (evidence).
Common Errors
• Writing about the experience and application
without referencing any theory, reflection, or
specific learning outcomes
• Skipping a learning outcome
• Not writing well – Portfolios are primarily for
students who like to write and/or have taken
English Composition.
• Failure to cite a source or provide enough
evidence of learning.
• The learning is not at the college level
(at a “C” grade or above).
Preserving Academic Integrity
Choosing the assessor:
• Assessor is a subject matter expert with
understanding of expectations for the level of
course credit being sought and understanding
of application in the field.
• Assessor has taught the course and has
been trained in portfolio assessment.
• Assessor uses the same rubric as the
student.
• Assessor provides feedback.
• Portfolio is assessed within a reasonable
timeframe by a trained faculty assessor.
An appeals process is available to the student.
Portfolio Assessment:
Faculty Perspective
Dr. Melissa Nemon
Portfolios (in My Experience)
• Typical structure:
– Personal experience
– Introduction to course
– Explanations by course
outcomes
Melissa Nemon, Ph.D.
• Great portfolios expertly
connect their professional
experience directly to a
specific course outcome
How Do You Assure Consistency?
Course Outcomes –
The more concrete the course outcomes are
in the syllabus, the more directly a
correlation between experience and course
expectations can be determined.
• This reduces bias
• Increases reliability
• And validity is ensured through a
standardized rubric for scoring the
portfolio
Traditional Course v. Portfolio
• In many cases, students have significant
experience in a particular course they are
seeking credit for therefore there does not
appear to be any loss of fidelity between
the traditional course and a portfolio.
• Issues that are avoided by portfolios
– Bored students in class
– Over-zealous students in class
– Contrarians in class
Assessing a Portfolio
• Takes about 2 hours to thoughtfully
assess a portfolio (depends on course
and supporting documentation)
– Looking for students to “prove” their
knowledge through expressions of
learning behavior
– How did they derive meaning from
specific experiences; how did they
learn what they now know and how did
they continue to use it; how do they
value learning?
Appeals and Partial Credit
• In my experience I have never overturned
an existing score based on an appeal
request – this speaks to the reliability of
the syllabus outcomes & CAEL rubric
• There is no partial credit because this is
not a paper for a class but instead the
whole class and much like you would not
give partial credit for a final grade, CAEL
does not assign partial credit.
So Why Do I Assess Portfolios?
• Vast experience with Military personnel
and Veterans returning to school
• Considerable experience with highly
English-proficient foreign students who
end up repeating courses due to
differences in degree alignment or
transfer status
• Great to connect academic proficiency
with practical knowledge – many students
do not make the connection between the
two but portfolios give us real examples
Using LearningCounts Portfolio Assessment Service
Kristen Himmerick
What is LearningCounts?
Comprehensive virtual end-to-end
service:
– Teach an online portfolio development
course
– Provide tools and guidance to
assist students through the portfolio
process
– Offer individualized assessment of
portfolios by qualified faculty
College Credit Predictor
CAEL 150 Instructor Supported Course
Course Name
CAEL 150 – Experiential Portfolio Development
Course Length
4-weeks in length
Description
This is an instructor-led, online course with weekly
assignments.
Credit Details
1 Credit Elective Pass/Fail
Grade Delivery
Evaluated by National College Credit Recommendations Service
Via Grade Form to MCSS College Registrar
Portfolio Development Process
Most Popular Portfolio Topics
•
•
•
•
•
•
•
Business and Management
Information Systems/Technology
Human Resources
Finance/Accounting
Criminal Justice/Legal
Hospitality/Event Management
Healthcare Administration
Assessment Process
• Portfolio assessment is $125/one course portfolio
• Portfolios contain a single course
• Must be undergraduate (100-400) level
– No capstones
– No internships
– No developmental courses
• Portfolio evaluations are completed in about two
weeks
o Every portfolio receives individualized attention
o Credit recommendations are Pass/Fail
Faculty Assessors
All faculty members are screened
for their credentials:
• 100% hold a minimum of a
Master’s degree in their field;
• Significant portion hold a terminal
degree;
• 100% work in their fields as
subject matter experts;
• 100% have taught/or are currently
teaching college-level courses;
• Receive ongoing training and
evaluation for continuous
improvement; and
• 100% are committed to our
PLA philosophy and student
success!
Student Portfolio Stories
LC Student Profile of Success
•
Is a strong college level writer and has taken or tested out of Basic Composition
Course
•
Has completed at least 1 other online course, has technical literacy skills and will
feel comfortable completing a course on another LMS
•
Feels comfortable writing a 5-10 page paper detailing their learning and
expertise on the course topics
•
Has several (at least 5) years of professional experience
in an applied area that aligns with college coursework
(communications, supervision, management, information
technology, marketing, healthcare industry, merchandising,
etc.)
•
Has a specific degree plan and has flexibility
within his/her remaining credits that could be
fulfilled by portfolio assessment credits.
•
Understands they may have to dedicate about
5 hours to complete their portfolio after
completing the portfolio development course
CAEL- MCSS Pilot Project
Goal: Build the capacity of Michigan community colleges
to increase degree completion rates using prior learning
assessment (PLA)
 MCSS and CAEL partnering on PLA pilot project
 LearningCounts online portfolio service at three Michigan
community colleges as a pilot
 Colleges will be competitively selected
 Funded by Kresge Foundation
Pilot Benefits to College
• LearningCounts implementation fee paid ($2795)
• LearningCounts Featured Network Annual Fee
($10,500) paid for 2 years
• On-site Faculty Assessor Training for up to 20
faculty members ($12,000)
• On-site and online prior learning assessment
training for Advisers and Admissions
• Marketing consulting and assistance for promoting
LearningCounts
Pilot Expenses
• College is invoiced $200 for each student
enrolled in CAEL 150
Cost to College Covered by individual college tuition & fees
$200/student
Book Fee, Instructor pay, Prior Learning
Assessment coaching to student, Technology
costs, Technology support
Tuition and fees could be paid by financial aid for
eligible students
• After completing CAEL 150, Students pay
portfolio assessment fee $125/portfolio to
LearningCounts
Sustainability After Pilot
•
Explore offering CAEL 150 through Michigan Colleges
Online
• MCO holds the Learning Counts License with colleges
contributing to the annual costs
• Establish revenue sharing with CAEL for tuition
Pilot Calendar – Year 1
April
Application Deadline April 30, 2015
Submit applications to
[email protected]
May
School Selection
June-July
On-site and Online Adviser Training
Marketing materials developed
August
Additional Online Adviser Training
September CAEL 150 Course Launches at
Colleges
October
Additional Online Adviser Training
Questions and Discussion
Questions:
How can I learn more about portfolio
assessment?
– Assessing Learning: Standards, Principles &
Procedures, 2nd edition, by Morrey Fiddler, Catherine
Marienau, and Urban Whitaker.
– Earn College Credit for What You Know, 5th edition,
by Janet Colvin.
– http://www.cael.org/pdfs/PLA_Fueling-theRace.pdf
Please type in your questions in the chat box in the
lower right portion of your screen.
Thank You!
Don’t Forget: Complete the application by April 30, 2015
Email completed application to [email protected]