Assessment
March–April 2013
Volume 25, Number 2
UPdate
Progress, Trends, and Practices in Higher Education
“Just Right” Outcomes Assessment:
A Fable for Higher Education
Catherine M. Wehlburg
T
here is an interesting paradox in the field of higher education assessment
of learning. Too much emphasis on assessment leads to “teaching to the test” and
an emphasis on getting the “right” answer. Too little emphasis on assessment leads
to no intentional directions and a meandering of learning. Let me take a moment to paraphrase a well-known fairy tale, taking many liberties with the original text.
Dr. Goldilocks is the provost at the well-known and highly ranked Great State University (GSU). GSU recently completed its last accreditation cycle. The visit went relatively
well, with some follow-up reports focused on assessment in the academic and academic
support areas. (There were also some problems with board policies, but those have been
rectified and aren’t the focus of this little story.) However, Dr. Goldilocks is in the unfortunate position of having to re-create GSU’s assessment processes because the university’s
last director had a nervous breakdown following the visit. So she is trying to reestablish
the accreditation office in a healthy way that will have a long-term, positive impact on the
institution. Dr. Goldilocks has been in dialogue with several university officials about this
issue and has determined that three different models offer ways forward.
The first model, recommended by the chair of the board of trustees, is to focus on
gathering information that is required by the state and the accreditors and that will demonstrate that GSU is well above average. (Actually, according to the board chair, GSU is
perfect and must have 100 percent achievement of all outcomes so that the results can be
sent to the press to extol the virtues of the mighty institution.) Dr. Goldilocks is concerned
about this model because it makes assessment data a hot topic of conversation, and the
results of assessment may actually burn some of the faculty.
The second model, recommended by the chair of the faculty senate with full approval
of the senate executive committee, is to leave assessment in the hands of the faculty,
knowing that faculty members know how to assess learning and that they will do it out of
the goodness of their hearts. Again, Dr. Goldilocks has concerns about this model because
she fears it will lead to a freeze on all assessment activities. While she fully recognizes
that faculty members do know how to assess learning, she isn’t sure they will continue
to do this without any administrative requirements. And, while faculty senate members
are sure that assessment activities will continue without supervision from an assessment
office, the thought leaves her a bit cold.
The final model being considered requires some oversight from a revamped office of
assessment but applies the results in a mostly formative way, allowing for assessment
data to be used both for accountability and improvement. Since this is a fairy tale, we
CONTENTS
ARTICLES
“Just Right” Outcomes Assessment: 1
A Fable for Higher Education
Catherine M. Wehlburg
Editor’s Notes
Linda Suskie
3
Focus on the Bottom-Line: Assessing Business Writing Michael Cherry, George Klemic
5
Including Students in Student Assessment
Eric Niemi, Carolinda Douglass
7
Using Curriculum-Embedded Assessments of Student Learning:
Establishing a Model for Internal
Benchmarking
Mitchell H. Peterson, Amanda
Gustafson
9
FEATURE
Assessment Measures
Gary R. Pike
View this newsletter online at
wileyonlinelibrary.com
doi:10.1002/au.252
© 2013 Wiley Periodicals, Inc.
12
Assessment Update
Progress, Trends, and Practices
in Higher Education
March–April 2013
Volume 25, Number 2
Editor
Trudy W. Banta, professor of higher
education and senior advisor to the
chancellor, Indiana University–
Purdue University Indianapolis (IUPUI)
Managing Editor
Karen Elaine Black, director of program
review, IUPUI
Assistant Editor
Frances W. Oblander, director of institutional
research & effectiveness, South University
In too many cases, assessment has, conceptually, been removed from
the area of teaching and learning and moved into the administrative,
accreditation area.
Book Review Editor
Susan Kahn, director of institutional
effectiveness, IUPUI
Consulting Editors
Peter T. Ewell, National Center for Higher
Education Management Systems
Thomas Anthony Angelo, La Trobe University,
Victoria, Australia
T. Dary Erwin, James Madison University
Cecilia L. Lopez, Harold Washington College
Marcia Mentkowski, Alverno College
Jeffrey A. Seybert, Johnson County
Community College
Peter J. Gray, US Naval Academy (ret.)
Gary R. Pike, IUPUI
Assessment Update: Progress, Trends, and Prac­tices
in Higher Education (Print ISSN 1041-6099; online ISSN
1536-0725 at Wiley Online Library, wileyonlinelibrary
.com) is published bimonthly by Wiley Subscription
Services, Inc., A Wiley Company, at Jossey-Bass, One
Montgomery St., Suite 1200, San Francisco, CA 941044594. Periodicals Postage Paid at San Francisco, CA,
and additional mailing offices. Individual subscriptions
are $135 per year (institu­tional $224). Back issues are
available in limited supply at $29 per issue. To order,
phone toll-free (888) 378-2537 or fax (888) 481-2665.
Visit our Web site at www.josseybass.com. Postmaster:
Send address changes to Assessment Update, JosseyBass, One Montgomery St., Suite 1200, San Francisco, CA
94104-4594.
Copyright © 2013 Wiley Periodicals, Inc., A Wiley
Company. All rights reserved. Reproduction or translation
of any part of this work beyond that permitted by Section
107 or 108 of the 1976 United States Copyright Act
without permission of the copyright owner is unlawful.
Requests for permis­sion or further information should
be addressed to the Permissions Department, c/o John
Wiley & Sons, Inc., 111 River St., Hoboken, NJ 07030;
(201) 748-8789, fax (201) 748-6326, www.wiley.com/go/
permissions.
2
can conclude that this was easily done with just a touch of magic and that everyone lived
happily ever after.
However, in the real world, most provosts don’t have a magic wand, and getting to happily ever after involves a lot of work, some compromise, and a recognition that assessment
is a journey and not a destination. But we can learn from this fairy tale—the type of focus
on assessment needs to be “just right.”
How do we make sure that assessing learning is just right? This is more easily done in
a department where the chair is focused on student learning and supports and understands
the need for the department to state outcomes clearly and then map the curriculum to
these outcomes. But this focus rarely exists across an entire university, much less across
multiple institutions. How can we scale assessment to this just-right stage while also mandating that it occurs in a realistic and sustainable way?
The mandates of assessment via accreditation standards have been successful in moving assessment to become a key focus in higher education. While some faculty members
and administrators may see assessment as a fad, at least it is a long-term fad. However, the
focus on assessment for accreditation has moved us in the direction of “too hot.” Much of
the assessment work in higher education is done in preparation for or as a response to an
accreditation visit. In too many cases, assessment has, conceptually, been removed from
the area of teaching and learning and moved into the administrative, accreditation area.
(continued on page 15)
Call for Contributions
The editor welcomes short articles and news items for Assessment Update. Guidelines
follow for those who would like to contribute articles on outcomes assessment in higher
education.
• Content: Please send an account of your experience with assessment in higher
education. Include concrete examples of practice and results.
• Audience: Assessment Update readers are academic administrators, campus assessment practitioners, institutional researchers, and faculty from a variety of fields.
All types of institutions are represented in the readership.
• Style: A report, essay, news story, or letter to the editor is welcome. Limited references
can be printed; however, extensive tables cannot be included.
• Format: In addition to standard manuscripts, news may be contributed via letter,
telephone, or fax (317) 274-4651. The standard manuscript format is a 60-space line
with 25 lines per page. Articles may be sent to [email protected] as a Microsoft
Word attachment. Please include your complete postal mailing address.
• Length: Articles should be four to eight typed, double-spaced pages (1,000–2,000
words). Annotations of recent publications for the Recommended Reading feature should
be 200–500 words in length. Short news items and content for the Memos section should
be about 50–200 words long.
• Copyright: Articles shall not have been registered for copyright or published
elsewhere prior to publication in Assessment Update.
• Deadlines: Each issue is typically planned four months before its publication.
Please address mailed contributions and comments to Trudy W. Banta, Editor,
Assessment Update, Suite 140 Administration Bldg., 355 N. Lansing St., Indianapolis,
IN 46202–2896. ■
Assessment Update • March–April 2013 • Volume 25, Number 2 • © 2013 Wiley Periodicals, Inc. • doi:10.1002/au
“Just Right” Outcomes Assessment: A Fable for
Higher Education
(continued from page 2)
This has worked to get assessment moving, but I believe that this approach has
reached the extent to which assessment is
used. We have to change the paradigm.
This new just-right paradigm will allow for the primary purpose of assessment
to be improving and enhancing learning
while requiring the documentation of results that will be used for accreditation.
The focus should be on assessment as
a part of the teaching/learning process
rather than something removed for accountability. This is, of course, more easily said than done. But if we don’t move
toward this structure of assessment, we
will forever be bouncing between too hot
and too cold.
Lee Shulman (2007) suggested that
assessment is a way to tell the story of the
department or unit. In Shulman’s metaphor of storytelling, assessment becomes
the way to build a narrative that can be
shared outside the department with those
who may not know the workings of that
area or discipline.
The story told by an assessment is . . .
ultimately a function of the dimensions of measurement that determine
the possible directions the narrative
might take. So accountability requires
that we take responsibility for the
story we commit ourselves to telling.
We must make public the rationale
for choosing that story as opposed to
alternative narratives. . . . [O]nly then
should we defend the adequacy of the
forms of measurement and documentation we employ to warrant the narratives we offer. (22)
The gathering of assessment data allows for sharing information—but we can
share only what we know. If we don’t ask
the right questions or identify the mean-
ingful goals and outcomes, the full story
can never be told. By thinking of assessment as a way to tell a story, data can be
shared. But the storytelling aspect of assessment doesn’t just have to be the story
told to the outside (and, ostensibly, for accountability purposes). That story should
first be told to the department so that it can
be used to enhance and improve teaching
and learning. This is the key difference
in the just-right assessment process—
improvement comes first. And, if done correctly, improvement based on documented
data can also be used to share with others
for accountability and accreditation.
In order to do this, though, there needs
to be a serious consideration of why we
do assessment as a department, an institution, and a system of higher education.
At this point, the most-often stated reason is that we are required to do so—if
we do not do it (or if we do it badly), we
are punished. Our institution may go on
probation from the accreditor, our provost or dean may fuss or remove merit
opportunities, or there may even be attempts at public shaming by publicizing
ment don’t do a lot for your tenure and
promotion dossier at most institutions.
And if you do take assessment of learning seriously and determine from your
data that there are areas for authentic and
candid improvement, you actually may
be punished for not sharing publicly how
perfect your department or institution
is. Because faculty members, staff, and
administrators are typically very smart
people, the current structure of assessment is to promote the creation of reports
that don’t really do much to change the
status quo—those versions of the assessment story that detail how the department
or institution is just about perfect but also
keep away the accreditors and the state
legislature by providing just enough data
to show that you are playing their game.
This paradigm wastes a lot of time and
resources and ends up being a frustrating (and never-ending) reporting cycle.
So, back to the question at the beginning of this paragraph: Why do we do
assessment?
Higher education should assess learning because this is how we improve. Assessment needs to be a transformative
process that looks at where we want to
go and how close we came. There needs
to be reinforcement for doing assessment
well—a reward structure not for “doing”
assessment but for actually transforming
Improvement based on documented data can also be used to share
with others for accountability and accreditation.
lists of those departments/units that have
not completed their assessment reports.
The point of all of this punishment seems
to be that nothing good comes from doing assessment. Getting your assessment
report done and turned in on time allows you to avoid the fussing or negative
feedback you will get if you don’t do it.
But what if you do assessment (and do it
well)? What rewards can you expect from
the institution? Usually there are none.
Or if there are rewards, they are fleeting
and minor. Those gold stars for assess-
based on good and meaningful data. If
this were a fairy tale, I would wave my
magic wand and instantly create a system
where just-right assessment is rewarded
and courses, programs, and institutions
all gather assessment data and use it for
improvement while sharing these stories
with society. But it’s not a fairy tale, and
changing higher education is difficult.
Nevertheless, we can change—it isn’t
easy, but it can certainly be done. Change
has to occur on several levels: course,
program, institution, and nationally.
Assessment Update • March–April 2013 • Volume 25, Number 2 • © 2013 Wiley Periodicals, Inc. • doi:10.1002/au
15
978-1-118-71456-0
Faculty members can, of course, use
assessment for improving their courses.
A lot has been written on this in the faculty development literature (e.g., Wehlburg 2006). Accreditation requirements
for assessment have focused historically
on getting assessment done, but now
many regional accreditors are focusing
more on using assessment data, and this
will help move us into the just-right assessment paradigm. One of the most difficult changes to be made is at the institution level. Faculty members face many
competing forces in higher education:
the need to teach, do research, provide
service, and keep up in their disciplines.
Since most institutions have focused on
assessment as a task to be completed for
the accreditor, this is how most faculty
have approached assessment. It has been
in the service area and not in the teaching realm. This is where the change needs
to happen in order for institutions to use
assessment effectively and just right. As-
sessment is not a service; it is how we
improve and enhance teaching and learning. Assessment offices should not be
stand-alone administrative structures. Assessment should be embedded in teaching and learning centers and should be a
part of the faculty role in educating students. After all, how will we know what
students are learning if we can’t or don’t
assess it?
If assessment does become embedded in teaching and learning, the reward structure needs to reflect this.
Faculty annual reports should ask how
the course and academic programs have
been improved and enhanced based on
assessment of student learning outcomes. Merit should be awarded because
teaching changes have been made based
on data collected in previous semesters.
And budgeting should reflect assessment
goals and processes. Why give additional money to a department that shows
it is meeting all of its learning goals?
Shouldn’t we focus on departments that
are not meeting these goals but have an
action plan to improve? By changing the
reward structure, we can influence how
assessment is used. And when it is used
in ways that enhance teaching and learning, it can be documented and used for
accountability. I’m sure that this is what
Dr. Goldilocks would like to see happen
at her institution. ■
References
Shulman, L. 2007. “Counting and Recounting: Assessment and the Quest
for Accountability.” Change 39: 20–25.
Wehlburg, C. 2006. Meaningful Course
Revision: Enhancing Academic Engagement Using Student Learning
Data. Bolton, Mass.: Anker.
Catherine M. Wehlburg is assistant
provost for institutional effectiveness
at Texas Christian University in Fort
Worth, Texas.
Copyright of Assessment Update is the property of John Wiley & Sons, Inc. and its content may not be copied
or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.
However, users may print, download, or email articles for individual use.