Is It a House... Or a Pile of Bricks? Important Features of a Local Assessment System
Author(s): Theodore Coladarci
Source: The Phi Delta Kappan, Vol. 83, No. 10 (Jun., 2002), pp. 772-774
Published by: Phi Delta Kappa International
Stable URL: http://www.jstor.org/stable/20440251
Accessed: 11/03/2010 15:05
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=pdki.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact [email protected].
Phi Delta Kappa International is collaborating with JSTOR to digitize, preserve and extend access to The Phi
Delta Kappan.
http://www.jstor.org
A SPECIAL SECTION ON ASSESSMENT
Is
It a House
Important
. ..
Features
or
a
Pile
of
of a Local Assessment
Bricks?
System
By addressing the six features thatMr.
Coladarci describes, school leaderswill
be working more deliberately towarda
true local assessment system, rather
than a mere collection of assessments.
BYTHEODORECOLADARCI
N 1996 theMaine
legislature established the Learning Results, a comprehensive set of standards to be
achieved by all students.' The legislators further decreed that student achievement of these standards
must be measured by a combination of state and local assessments. Toward this end, Maine school lead
ers now face the daunting charge to develop "local assessment systems." It is not yet clear whether the
Maine Department
of Education will subject a district's assessment system to procedural review and
approval, but the stakes are high nonetheless: by 2007, a high school diploma inMaine will be tied to
achievement of the Learning Results, and local assessment systems are to provide the means for certify
ing thisachievement.
To be sure, the welcome upside of these develop
ments is that school leaders inMaine are being asked
tome/ state and local achievement information for cer
This synthetic
tifyingachievementof statestandards.
approach runs counter to the practice in 18 states
soon to be 24
where high school graduation is linked
inextricably to passing a state test.2However, a clear
downside of theMaine policy is the absence of clari
ficationfrom the state legislature, from the experi
THEODORE COLADARCI is a professor of education, College
of Education and Human Development,
University of Maine,
Orono.
772 PHIDELTA
KAPPAN
ence of other states, or from the professional literature
regarding just what a local assessment system is.
While the term "local assessment" isclear enough, "as
sessment system" is often used in reference to a states
-
Com
testingprogram.Forexample,theMassachusetts
state
prehensive
AssessmentSystem,a standards-based
test, is administered to all students in grades 4, 8, and
10.' So again, justwhat is a "local assessment system"?
In particular,what makes such a system, in fact, a system?
This is the fundamental question with which Maine
school leadersarecurrentlywrestling.
The Maine Department
of Education
formed an
Colvin
Illustration:
Artvi//e
Collection/Rob
advisory group, of which Iwas amember, to deliber
ate this question. Fueled by our resolve to provide
helpfulguidelines to school leadersand informedby
the thoughts of others,4we identified what we believed
to be important features of a local assessment system.
Insofar asMaine isnot alone in calling for the creation
of such systems, our observations may provide guid
ance for school leaders in other states aswell. Indeed,
asmore and more states begin to revisit the advisabil
ity of single-test graduation policies,5 the notion of an
assessment systemmay be seen as an increasingly at
tractive alternative to high-stakes testing. This is now
even more likely as a consequence of theNo Child Left
Behind Act, which allows states to use a combination
of state and local assessments in satisfying the require
ment for annual testing in grades 3 through 8.'
WHAT
ISA LOCAL ASSESSMENT
SYSTEM?
A "system,"theAmericanHeritageDictionarytells
translation of the general to themeasurable, however,
assessmentisproblematicat best.
Second, a local assessment system ismade up of as
sessments that are initiated at the classroom, school, dis
trict,and statelevels.Classroom-levelassessmentsre
flect the day-to-day evaluation practices of teachers,
such as running records, unit exams, papers, projects,
andportfoliosofwork samples.School
performances,
and district-levelassessmentsinvolvestudentsacross
multiple classrooms.Forexample,adistrictmight ad
minister a reading proficiency test to all second-grad
ers or a science proficiency test to all eighth-graders.
What is the role of a state-mandated test in "local"
assessment
Assessment
TheMaine Educational
systems?
(MEA), aligned with the Learning Results and admin
istered to allMaine students in grades 4, 8, and 11, is
an important source of evidence regarding local progress
toward achieving the state standards.While "MEA"
arearguably
and "localassessment"
contradictory
terms,
in a system must cohere. Thus a local assessment sys
tem is a coherent, coordinated plan for assessment. It
is a constellation of measures that, together, yield data
that document progress toward student mastery of an
"MEA" and "local assessment system"are not. Thus a
valid state-mandated test canand shouldbe con
sidered an element of a local assessment system.
Third, theassessmentsare conducted inmultiple grades.
Classroom-level assessments, of course, are conducted
in all grades, whereas school-, district-, and state-level
assessments are most likely administered to selected
nounced learningtargets.Clearly,thesystem ismade
grades.Only by consideringachievementinformation
up of individual assessments. But a collection of assess
ments does not entail a system any more than a pile of
acrossmultiple grades can one monitor local progress
toward student mastery of the learning targets.
us, is a group of interacting, interrelated, or interde
pendentelementsforminga complexwhole."Elements
bricksconstitutesa house.Therefore,the fundamental
question for school leaders is this: Inwhat sense does
their plan constitute a systemof assessments, rather than
amere collection of assessments?
In the view of the advisory group, a local assessment
system has at least six important features. First, the as
sessmentscollectively are relevant toannounced learning
targets.That is, a local assessment system provides evi
dence of student achievement regarding formally speci
fied, rather than tacit or implied, learning targets. Fur
ther, these learning targets are stated with sufficient
Fourth, theassessments
drawonmultipleformats
"traditional"and "alternative"alike. There are various
ways to appraise student learning, such as a selected
responseformat(e.g.,multiple-choice,matching, or
true/falseitems),a constructed-response
format(e.g.,
specificityto communicatemeasurableoutcomes
outcomes that are directly amenable tomeasurement
and assessment.
The prefatory"guidingprinciples"
of
theMaine Learning Results (e.g., "Each student must
leaveschool as a clearand effectivecommunicator")
areexamplesof goals that lackthe requisitespecificity.
To be sure,measurableoutcomescanbe derivedfrom
such statements.InMaine, this task isaccomplished
by thedelineationof "performance
indicators"
foreach
content standardin theLearning
Results.
Without this
All thoseyearsof skipping
phys. ed. finallycaught
upwithRodneyBeltram.
JUNE 2002
773
workedproblems,shortanswers,essays),andperform
No one
ancemeasures(e.g.,projects,demonstrations).
method issufficientforallpurposes.For example,se
lected-responseitemsarearguablysuperiorto either
or performance
measuresforas
constructedresponses
sessingrecalland basicunderstandingof a largebody
of content,whereas the lattertwomethods areprefer
expression.
ableforassessing
written,oral,or behavioral
will doubt
Insofaras theannouncedlearningtargets
lessrepresentavarietyof outcomes,a localassessment
systemshouldcomprisea varietyofmeans forassess
ing thoseoutcomes.
systemallowsfor
Fifth,a localassessment
multipleop
and
knowledge,
portunitiestodemonstrate
understanding,
A singleadministrationof an assess
skilldevelopment.
ment, whatever its form, typicallyprovidesan insuf
ficientbasisformaking inferencesaboutstudentpro
ficiency.Inferencesaremore defensiblewhen students
havemultipleopportunitiesto demonstrate
knowledge
a
assess
For
andunderstanding. instance, performance
ment canbe conductedat severalpoints in time,or the
same learningtargetscan be assessedthrougha com
binationof assessmentformats.
in thesystem
hasan announced
Finally,eachassessment
rationale.In particular,the assessment'spurpose,au
with other assessmentsin the
dience,and articulation
systemshouldbe clearlystated.For example,perhaps
assessments
theannouncedpurposeof classroom-level
de
is tomonitor achievementandguide instructional
cisionson a day-to-daybasis,with studentsand par
ents servingas theprimaryaudience.As for theirar
ticulationwith other assessmentsin the system,class
might be seenasyieldingmore
room-levelassessments
information
about
detailed,nuanced,andcontextualized
studentachievementthan, say,district-or state-level
assessmentscan be expectedto provide.
For anotherexample,considera readingproficien
cy test that a schooldistrictadministersannuallyat
theend of grade2. Here, the formativeevaluationof
the readingprogramisperhapsthe statedpurposeof
this test,while theaudienceisprimary-grade
teachers,
schoolboardmembers,and thegeneralpublic (which
Incom
suggeststhe relatedpurposeof accountability).
statetest,thedistrict'sread
parisonto thefourth-grade
ing testmight be seenasprovidingamore comprehen
siveportraitof a student'sreadingproficiencyand at
amore criticalpoint in development.Also, given the
announcedpurposeof this test-program evaluation
would be seen as an im
- its "standardized"
natureportantcomplementto theachievementinformation
774
PHI DELTA KAPPAN
thatderivesfromclassroom-levelassessments.
Although the assessmentsthatconstitutea system
differin theirannouncedpurposes,audiences,and ar
ticulation,the individual
assessments
do not existin iso
lation.Each shouldbe used by educatorsto confirm
theirinferencesand conclusionsfromothermeasures
in the localassessmentsystem.
FURTHERCONSIDERATIONS
In addition to the six featuresjustdescribed,there
are technicalissuesthatschool leaders
must consider
when developingandmonitoring localassessmentsys
tems.7
Although a detaileddiscussionof these issuesis
beyond thescopeof thisarticle,ingeneraleachassess
ment in the systemmust be of demonstrablevalidity
and reliability.
That is,eachshouldmeasurewhat it is
supposedtomeasure,and it shoulddo so consistent
ly.Further,interpretation
of assessmentresultsshould
be guidedbydearperformance
standards.
Finally,these
technicalconsiderationsare importantfor the system
as awhole, aswell as for the individual
measures that
it comprises.
The making of a localassessmentsystem requires
The
considerablethought,effort, time,and resources.
systemisnot establishedquicklyand inone fellswoop;
it evolves.By addressingthe six featuresabove,school
leaders
will beworkingmore deliberatelytowarda true
assessmentsystem,ratherthanamere collectionof as
sessments.
1. State
ucation,
htm.
Maine
Learning Results (Augusta: Maine
Department
of
at
1997). Available
http://www.state.me.us/education/lres/lres.
of Ed
2. Lynn Olson,
"States Adjust High-Stakes
Testing Plans," Education Week,
on
pp. 1, 18-19; and idem, "Overboard
January 2001,
Testing?,"
Education
28-30.
2001, pp. 23, 25-26,
Week, 7 February
24
3. For an overview
of MCAS,
see
http://www.doe.mass.edu/mcas/aboutl.
html.
A Guide
4. Judith A. Dawson,
Assessment Handbook:
As
for Developing
sessment
in
Illinois
Schools
Illinois
of
Programs
Department
(Springfield:
Assessment
Education,
Land, "Moving Up to Complex
1995); Robert
from the 1996 CRESST
Evaluation
Conference,"
Systems: Proceedings
Forum on Assessment,
1997, pp. 1-21; National
Principles
and Indicators for Student Assessment Systems (Cambridge, Mass.:
FairTest,
Assessment
Coordinated
1995); and Edward D. Roeber, Designing
Sys
Comment,
temsfor Title I of theImprovingAmericas SchoolsAct of 1994 (Washing
ton, D.C.:
Council
5. Olson,
"States Adjust
6. Lynn Olson,
Week, 6 March
7. Theodore
of Chief
State
School
High-Stakes
"Testing Rules Would
2002, pp. 1, 36-37.
et al., Measured
a Local Assessment
Coladarci
tions for Developing
ment
of Education,
2000).
IC
Officers,
Testing
Grant
States Leeway,"
Measures:
System
1996).
Plans."
Technical
(Augusta: Maine
Education
Considera
Depart