Critique: Assessment and the Production of
Learning
ELISABETH SOEP
University of California, Berkeley
Education researchers and classroom teachers have argued that the constant pressure
to measure and rank students makes it difficult to shape assessment as an episode of
learning. Yet we know little about how learning moves in and through assessment of
any kind. Building on two national multisited studies, the research reported here uses
ethnographic techniques to examine learning within critique. Critique is a form of
assessment through which young people jointly judge their own work and that of their
peers. The article focuses on episodes of critique within two nonschool sites for collaborative production involving ethnically and economically diverse groups of
youth—a community-based video project and an organization in which young people
create radio stories for local and national broadcast. Learning environments such as
these draw voluntary youth participation and are organized around sustained projects
released to outside audiences. Findings indicate that critique manifests itself as an
episode of learning by engaging young people in joint assessment events that are
improvisational, reciprocal, and oriented toward the future of the work under review.
Intra- and cross-site comparisons suggest that critique is likely to arise within specific
conditions: when stakes are intense, metastandards are subjected to review, accountability is mutual and interactively sustained, and interdisciplinary practice is mandatory. The article reviews various ways to conceptualize learning and argues in the
end for a theory of learning as production, a way of making. Implications include
new ideas for research methodologies and new understandings of youth-adult collaborations in learning and production.
Inside any given classroom within the United States, disparate forces, including state standards, federal policies, legally mandated procedures, and
commercial products, shape even tiny moments of student evaluation.
Criticisms of classroom-based evaluation conventions are well established in
the education literature. Too much emphasis centers on measurement and
ranking and too little on how assessment can serve as an episode of learning
that is consistent with the larger pedagogical goals and practices of a given
classroom, while also attuned to the needs and dispositions of individual
children and youth.
Teachers College Record Volume 108, Number 4, April 2006, pp. 748–777
Copyright r by Teachers College, Columbia University
0161-4681
Critique: Assessment and the Production of Learning
749
Despite these criticisms, the current sense of urgency around school accountability makes it very difficult for even the most sensitive educators to
modulate how they evaluate their own students. Standardized testing procedures privilege discrete bits of knowledge over performances that reveal
active modes of knowledgeability. Although theorists may conceptualize
learning as inquiry or a process of situated participation, arguing that minds
develop through active engagement with meaningful tasks in specific contexts, rarely do mandated high-stakes evaluations focus on actual participation in anything other than a testing situation. Many reforms talk of
assessment as instruction, but the fact remains that we know very little about
how learning moves in and through assessment of any kind. The research
reported here examines learning within critique, a form of assessment
through which young people jointly judge their own work and that of their
peers as they prepare that work for public release.
ASSESSMENT AS A MOMENT-TO-MOMENT ACTIVITY
Part of the difficulty in assessment reform stems from the fact that complex
and collaborative learning tasks do not lend themselves to straightforward,
easily replicable evaluation methods; such tasks require the production of
new knowledge and experimentation with received ideas. It is not easy to
track and compare student learning when young people work together on
creative problem-solving projects. Evaluation under these conditions transpires not only through external appraisal but also, crucially, by way of
ongoing self- and peer-assessment by participants in the activity itself. Perhaps we should be looking, therefore, inside learning events, to the discourse of young people themselves, as we develop new strategies for
assessment. Support for this line of argument is revealed in research centered on assessment as a moment-to-moment collaborative affair. For example, in a recent study of young people involved in group work
assignments, Cohen and her colleagues (2002) found that the more students evaluated their own work as it developed, using criteria presented at
the beginning of the task, the more motivated and task focused was their
discourse, resulting in higher quality products and more sophisticated
written reflections.
Empirical evidence supporting the value of self- and peer-assessment is
critical in efforts to reimagine the role of evaluation in education. Stated
simply, it appears that the act of assessment itself is too often not sufficiently
participatory. Participatory evaluation would regularly place young people
in situations in which they must evaluate their own work and that of their
peers to move the work toward completion and ultimate release. Research
on how assessment messages circulate in everyday discourse beyond
750
Teachers College Record
classrooms advances this view. In traditional classroom formats, teachers
tend to control assessment, with students playing at best a passive role
(Cazden, 2001; Filer, 2000). In everyday conversation, by contrast,
assessment is often deeply participatory (Goodwin & Goodwin, 1992;
Linde, 1997). When one speaker utters an assessment in the course of
conversation, subsequent speakers tend to position themselves with respect
to the appraisal that they have just heard, by conveying agreement or expressing dissent.1 This process can be highly energetic, evident in flurries of
overlapping, densely populated stretches of speech, as in a phenomenon
especially conducive to collaborative problem-solving that Tannock (1998)
identified as ‘‘swarming.’’ In this sense, self- and peer-assessment can invite
heightened participation and represent a real educational opportunity to
engage students more fully in their own learning.
Young people themselves, when faced with certain kinds of learning
tasks—specifically, those aimed toward the release of original, collaboratively produced material to an outside audience—are constantly developing
the very criteria that they will then apply to their work as they evaluate its
merits. This activity can be described as a process of critique—the joint
assessment of original objects and performances by producers themselves.
CRITIQUE IN THE ARTS
Models for critique exist in every field: Scientists evaluate one another’s
theories as they prepare for meetings and clinical trials, just as architects
formally review design plans before pitching accounts, athletes screen game
tapes to recognize and learn from the lessons of past performance, and
political teams endlessly debate the nuances of single words when composing speeches and slogans. However, conventions for critique derived from
the arts are especially relevant, given the highly established, if underrecognized, rituals for self- and peer-assessment that occupy a prominent place
in arts education. Nonschool spaces for arts production in particular reveal
how young people use critique in settings that draw voluntary youth participation—contexts organized around sustained creative projects culminating in public release of young people’s work. A moment of critique in
these kinds of settings might involve a group of young people who are
producing a video and stop in the middle of a shoot to assess and radically
revise the choreography for a scene that is just not working as it was originally imagined. Or picture a group of teen radio producers preparing a
special series on youth, money, and marketing for a national show. They are
discussing the subtleties and controversies embedded within a spoken-word
poem written by a 17-year-old as they consider the poem as a possible
element in that series.
Critique: Assessment and the Production of Learning
751
These specific moments of critique took place within two different sites
for collaborative arts production: a youth video project called Cutaway, in
which young people produce movies screened in cinemas and museum
spaces, and a nonprofit media program called Youth Radio, in which young
people collaborate with adult producers on stories for local and national
broadcast.2 Representative episodes of critique in these settings contain
broader implications for the design of participatory, assessment-rich learning environments. Analysis of these interactive moments begins with a
definition of assessment as a process of seeing and responding to a given
piece of work, and critique as a practice of mutual assessment that young
people use when they turn to one another as resources for judgment. After
a brief review of literature on formal peer critique conventions in arts education, and a clarification of the conception of learning being applied and
advanced here, the analysis is organized around four questions:
1. How does critique manifest itself in the process of moment-tomoment interaction? This question centers on the ways in which critique
provides access to specific learning opportunities by giving young people
responsibility for developing and applying judgment.
2. What are the implications of critique for learning practice? This question draws attention to the conditions that promote critique.
3. What are the implications of critique for learning theory? Response to
this question suggests a conception of learning as production.
4. What are the implications of critique for methodologies of research on
learning? This final line of inquiry suggests that models for critique
among youth contain lessons for the conduct of participatory, collaborative research.
RESEARCH ON ASSIGNED PEER ASSESSMENT
There is a strong, though underrecognized, tradition within visual arts education, especially on a postsecondary level, of including learners in the
assessment of their own projects and those of their peers through something called a crit.3 A crit is a period of time, usually several hours, set aside
during an art class when students display work and discuss with instructors
and one another the strengths and weaknesses of each piece. Any mention
of crits to a group of practicing artists is likely to invite a spirited response.
When told about my interest in crits as sites of learning and assessment, one
painter recalled his professor hurling a beer can at his painting during a
crit. Another watched in dismay as an instructor tossed student paintings
out of a high-rise building window. I have had my own difficult, although
752
Teachers College Record
admittedly less dramatic, experiences in crits—for example, waiting miserably until my monoprint was picked last in a crit in which students decided in what order to consider individual pieces of work. It felt like being
the last kid chosen for the kickball team in gym class.
Crits evoke other, highly positive narratives as well. Research based in
postsecondary settings reveals that many artists credit crits for motivating
their productivity by introducing intense deadlines, exposing them to new
techniques and interpretations, forcing them to refine their habits of perception as related to production, and building their fluency in vocabularies
of art (Soep, 2000). Unlike criticism, in which outsiders to an art-making
experience describe and evaluate a given piece, in crits, artists themselves
make judgments about their own evolving work, and they find language to
describe what they see, like, and dislike in the efforts of others who are
learning alongside them.
There is relatively little attention, however, to crits in education
scholarship (Bulka, 1996). The research that does appear tends to be instrumental, highlighting features of effective peer critique. These studies
often focus on university settings, calling for crits marked by a number of
auspicious characteristics, including discussion driven by aspiration
rather than ideological position (Rogers, 1996); close focus on specific
qualities of the work under review (Roth 1999); a climate of ‘‘spontaneity,
empathy, and equality’’ (James, 1996, p. 153); and a commitment to
‘‘provisionalism’’ in commentary offered in critique (Wernik, 1985). When
these qualities are not in place, research suggests that crits fail to inform
artists’ practices and judgments in any meaningful way, while making participants feel as if they are ‘‘being hung naked on the wall and ridiculed’’
(Wernik, p. 194).
Even fewer researchers have published refereed studies of critique in
K–12 visual arts classrooms. Barrett (1988, 1994, 1997, 2000) is perhaps the
most notable exception (also relevant are Blythe, Allen, & Schieffelin Powell, 1999; Cotton, 1981; Ende-Saxe, 1990; House, 2001). Barrett said that
crits among young artists should have a purpose, invite participation, include positive and negative reasoned judgments, emphasize interpretation
over evaluation, and privilege the viewer as the most important participant.
Facilitators of crits should be prepared, said Barrett, to ask and elicit good
questions and to address fundamental issues pertaining to intent, content,
subject matter, form, relationship of media to materials, sources of artistic
influence, and social issues. Barrett’s work and that of others in the research
literature provide a useful conceptual framework and a practical set of
principles for educators who facilitate crits in their classrooms.4 The existing literature also exposes some gaps in our understanding of the processes
of self- and peer-assessment as a property of learning. Specifically, we
know very little about how critique materializes spontaneously within
Critique: Assessment and the Production of Learning
753
collaborative undertakings when it is neither scheduled, nor staged, nor
assigned by anyone but young people themselves out of some kind of tacit
recognition that joint assessment is in order.
CONCEPTIONS OF LEARNING THAT INFORM AND ARISE
FROM CRITIQUE
Moving from assigned crits to emergent critique entails a shift in focus from
preplanned feedback sessions to moments of judgment brought on by the
demands of collaborative production. This shift requires a working conception of learning, or, more specifically, what Cohen and Ainley (2000) called a
‘‘theory of cultural learning’’ (p. 92). Such a theory accounts for how young
people study and develop ‘‘school knowledge’’ and how they ‘‘learn to
dance, or knit, or make love, or ride bicycles, or horses, or play football, or
write graffiti or poetry, or tell jokes, or tall stories, practice safe sex, use
computers, play musical instruments, conduct experiments’’ (p. 92).5
One well-established tradition for theorizing learning is by way of acquisition models, which frame learning as the internalization of discrete
information that is then transferred to new contexts. Students memorize
dates, study rules and patterns, and master terminology and habits associated with a given task or field, and then demonstrate those achievements
through systematic examination. Although this view may be intuitively useful, contemporary learning theorists have raised concerns about acquisition
models, arguing that they make learning passive and equate knowledge to
an exchangeable commodity. Nevertheless, it has been argued that the idea
that students acquire and eventually transfer information continues to
dominate the assessment literature, at least implicitly, to the extent that
researchers do not always fully develop alternative theories of learning that
drive their analyses of assessment conventions and proposals for change
(Delandshere, 2002).
On the opposite end of a theoretical spectrum from acquisition models is
a view that learning takes place simply by virtue of being alive and adapting
and adjusting to new conditions. Learning from this perspective is an
everyday affair—a way of handling the world and what happens within it. It
is less a bounded phenomenon that can be neatly isolated and tested than it
is a natural product of being in the world, taking in new information and
responding to it, all the while registering the ‘‘lessons’’ that arise out of lived
experience. This view is useful to the extent that it frames learning as an
ordinary practice, providing an important corrective to the tendency
to locate learning exclusively within settings where obvious and deliberate
instruction is taking place. Perhaps this conception is therefore a necessary step in opening researchers’ eyes to the existence and significance of
754
Teachers College Record
learning environments within young people’s families, peer groups, neighborhood streets, and media consumption practices, and in school classrooms. This view is, however, so vague and far reaching as to have limited
analytical utility.
A third view reconciles an interest in learning as a process of acquiring
information, with an interest in learning as a property of everyday experience, while at the same time offering ways to account for the creation of
new knowledge. This third view says that learning is a process of situated
participation, in which new information enables people to reorganize their
relationships with one another and with respect to their joint projects. In
other words, learning means shifting the conditions that shape minds, circumstances, actions, and products. In this situated view, knowledge is coconstructed within communities of practice (Chaiklin & Lave, 1996;
Greeno, 1998; Lave & Wenger, 1991; Papert, 1991; Rogoff, 1994; Varenne
& McDermott, 1999; for recent critical discussions, see Anderson, Reder, &
Simon, 2000; Fox, 2001; Hacking, 2000; Sfard, 1998; Young, 2000). This
notion of learning is a useful starting point, although this view will not be
where the analysis ends. Study of critique pushes this situated participation
model in a new direction—toward a view of learning as production, a way of
making. Conceiving of learning as production provides new theoretical
tools to help us work through some of the questions that the situated participation perspective leaves open—specifically, how learning migrates from
one situation to another or, more to the point, how a single situation can
itself contain multiple imagined contexts. Critique is a key process whereby
learners produce these imaginary contexts by conjuring hypothetical
scenes, responses, and consequences for the decisions made within the
process of production. In this sense, as we will see, assessment can do more
than measure learning. It can serve as the fuel, so to speak, that moves
learning forward beyond a single situated moment in time.
SITES AND SCENES OF CRITIQUE
This argument for a particular way of theorizing learning surfaces from
scenes of interaction within two nonschool sites for creative media production, Cutaway and Youth Radio. A focus on settings such as these is consistent with a growing interest within the field of education in learning,
assessment, and youth development beyond school walls (cf., Campbell,
2001; Heath & McLaughlin, 1993; Hogan, 2002; Hull & Schultz, 2001,
2002; Mahiri, 2003; Noam, Biancarosa, & Dechausay, 2002; Paley, 1995;
Soep, 2002). Research in this area acknowledges that young people spend
only a relatively small percentage of their time inside classrooms and that
the best community-based projects seem able to draw active, committed,
Critique: Assessment and the Production of Learning
755
and highly impressive performances of learning among youth. Language
development appears to extend well into later childhood and adolescence
through extended involvement with community-based projects over sustained periods (Heath, 1998b).
These observations do not, of course, mean that schools matter less,
only that life beyond school has the potential to reveal new insights relevant
to the design of learning environments likely to draw youth engagement
and high quality performance. The same students who might struggle
in a remedial English class write and perform spoken-word poems that
bring critical audiences to their feet. Young people with sporadic attendance
in classrooms or those who have dropped out of school entirely will
consistently attend nonschool projects such as media programs, theater
collectives, or aerosol art crews, moving into leadership positions and
making long-term commitments, advancing their own skill levels while
pushing the arts that they practice in new directions. Evidence of youth
engagement and achievement along these lines is reported in two national
studies that greatly inform the present analysis, through my own participation in those studies as a researcher and ethnographer. The first
is Harvard University’s Project Co-Arts, a study of pedagogy, assessment,
and neighborhood-based development within community arts centers in
low-income settings (Davis, 1998; Davis, Soep, Maira, Remba, & Putnoi
1993). The second is a Stanford University-based study (1987–1997)6 of
youth learning and leadership in community-based organizations, which in
1995 began to focus on arts-based sites, given the especially notable contexts
and outcomes for learning that these settings appeared to provide (Heath,
1993, 2001; Heath & Ball, 1993; Heath & Soep, 1998; Soep, 1996, 2000,
2002, 2004, in press).
The organizational and interactive features linked to effective
community-based youth organizations are well documented in the existing
literature—for example, opportunities for intensifying youth participation
and leadership, and sustained projects organized around cycles of planning,
preparation, practice, and performance (Heath & McLaughlin, 1994a,
1994b). Some investigators urge schoolteachers to forge relationships with
community projects, suggesting that classroom-based educators might
profitably model some aspects of the kinds of learning environments that
take hold more commonly beyond school walls. Others point to the considerable structural barriers that make these kinds of relationships very
difficult, given the compulsory, highly regulated character of education in
school. Overall, it seems important, as we pay more and more attention to
community-based learning environments for youth, not to imply that an
impenetrable boundary separates them from schools (Hull & Schultz,
2002). Such a boundary does not always exist for young people themselves,
and innovative educators in both realms of practice very often share a
756
Teachers College Record
commitment, and also the capacity, to create conditions in which youth
serve as producers and judges of their own development (Weis & Fine,
2000).
Moreover, researchers are increasingly reckoning with the dangers of
romanticizing nonschool learning environments as some kind of educational
panacea. It is all too easy to portray these places as ‘‘free’’ spaces where
young people can escape (or, more problematically, be rescued from) the
forces of inequality that they face in other institutional settings, including
school (Hull & Schultz, 2002). Hogan (2002) identified the ways in which
power dynamics can undermine community-based learning by shutting
youth out of meaningful involvement, limiting their participation to menial
tasks and potentially doing more harm than good, as yet another example of
an experience whose reality looks quite different from the ways in which it
was initially presented to youth participants. Trend (1997) considered artsbased community projects in particular, pointing to the striking unevenness
among such projects in terms of product quality and the depth of youth
engagement; his analysis suggests that young people are often invited into a
project to provide autobiographical content and access, but they may have
little say over how, where, and to whom the work is actually presented. In
her study of a youth media project, Fleetwood (2005) sharpened and extended this point about the limits to youth involvement in community-based
learning, and she cautioned that such projects have a tendency to pursue the
fantasy of an authentic youth voice, which itself often embodies a sensationalized portrayal of racialized ‘‘urban’’ youth experience.
What is striking in the literature highlighting the positive potential and
the pitfalls of nonschool learning environments is the extent to which assessment seems to be a kind of pivot point. Effective community-based
projects engage young people in the process of judging their own work and
that of their peers, and indeed this level of participation in assessment is a
hallmark of high-quality sites (Heath & Soep, 1998). At the same time, many
of the problems that can arise with these projects are revealed in the ways in
which young people are excluded from the process of formulating judgments and acting on them—when they are called upon, for example, to tell
their stories in graphic detail but then dismissed when it comes time to
make editorial decisions and transform confessional material into refined
works. It is therefore especially relevant, given these current debates in
educational practice and research, to look closely at the assessment practices
that take place within such projects, to illuminate how learning happens
there and how learning can be theorized in a broader sense. It is important
to recognize that critique finds no easy path into school processes and
products, for the institution is not set up to take assessment from students
into account within high-stakes measurements such as standardized tests.
Nevertheless, classroom educators who practice performance-based
Critique: Assessment and the Production of Learning
757
approaches to assessment within the everyday flow of their teaching regard
evaluation as an episode of learning. They create ‘‘real-world’’ events in
which young people perform ‘‘mastery’’ of a given subject for outside audiences and panels, and in preparing for these performances, students often find themselves in a position to critique their own work and that of their
peers (Darling-Hammond, Ancess, & Falk, 1995). Daily assessment inside
schools is a social practice in which students can be active agents (Filer,
2000); critique is therefore a powerful resource for teachers who see young
people as partners in the production of learning.
The research on nonschool learning environments reported in this article has evolved through two phases. Building on the national studies noted
previously, the first phase, from 1991 to 2000, included study of youth
learning within more than 20 community-based organizations, including a
teen drum corps, an improvisational theater troupe, a mural and graphic
arts project, a zine-writing group, and a video production project (Cutaway)
in cities in New England, the southeastern United States, and on the West
Coast. These last two sites were studied in particular depth for comparative
purposes, including, in the case of Cutaway, field work over 1 year that
entailed more than 190 hours of intensive data collection and tape recording to yield a language corpus for discourse analysis.7 This phase of the
research, although centered on projects facilitated by adult leaders, focused
especially on moments of critique among youth as they formulated judgments of their own work and that of their peers. My own participation as a
researcher in these moments of critique was minimal because it would have
seemed problematic, given the nature of the research, for my own habits of
critique to shape the phenomenon central to the study. In the second phase
of the research, from 2000 to the present, this emphasis on observation with
little participation gave way to a different arrangement entirely. Phase two is
an action-research project within one youth media organization, Youth Radio, where I work actively with young people to produce radio stories for
local and national broadcast. My own voice is, therefore, one among many
in the exchange of critique. This phase of the research looks more closely at
the workings of critique between young people and adults who are mutually, if asymmetrically, engaged in a project for which both parties will be
judged once the broadcast airs to significant audiences.
The analysis offered here focuses on one episode of critique from each
phase of the research, the first based at Cutaway and the second at Youth
Radio. These episodes were selected for their representativeness; these
were not random, nor were they exceptional occurrences. There were, to
be sure, long stretches of time within both research sites when no critique
among youth could be heard: when instructors lectured on a particular
topic or demonstrated a new piece of equipment, or when young people
worked silently in a studio or edit suite. Episodes of critique such as the ones
758
Teachers College Record
that follow predictably arose when the process of production itself required
a moment of display or performance—when a young video maker had to
present her script to the actors who would be playing the parts, or when a
radio producer was recording his narration in an announce booth and the
peer who was engineering that session provided feedback on voicing technique. These ‘‘naturally occurring’’ assessment junctures took place over
the entire course of a production project, from the earliest stages of idea
generation, through the final hours before the work would be released, to
the aftermath of an exhibition or broadcast. It grew especially intense when
some kind of conflict arose—when participants needed to make a crucial
decision—and there was disagreement as to the best way to proceed. Within
these kinds of moments, critique made its strongest appearance as a resource for identifying and solving problems through a process of assessment, in this sense significantly repositioning assessment as a practice that
fuels, rather than merely measures, learning. It is important to clarify,
however, that conflict is defined not necessarily as an acrimonious encounter, but instead as a moment when young people face some obstacle in
moving their work forward and when there is no immediate consensus on
how to proceed, nor is there an adult who single-handedly can make that
judgment on their behalf.
SCENARIO 1: CRITIQUE AMONG YOUTH AT CUTAWAY
Founded in 1995, Cutaway is located in a major West Coast city and involves
primarily working- and middle-class youth of color and led by a White artist
from a working-class family.8 In the specific Cutaway project highlighted
here, 8 young artists ranging in age from 14 to 18 spent one summer and
fall producing an original movie, acting and serving as crew members on
one another’s productions. Their individual videos were ultimately projected simultaneously in a video installation, meaning that they had to make
sense as standalone narratives and also as part of a total visual environment
in the gallery where the work premiered. While learning the basic technical
skills of video production, the young people spent several weeks brainstorming ideas for their individual movies and narrative and visual themes
that would resonate across all the work, lending coherence to the overall
installation. Lila, the adult artist in charge, launched the project with a loose
suggestion that perhaps the group might want to do movies that related in
some way to their neighborhoods, but by the time the young people had
developed scripts and begun shooting, they had diverged considerably
from that initial point of departure. There was one mockumentary about
flirtation rituals between boys and girls, an experimental documentary on
public art, and an earnest, soap opera-like love story about romance across
race and class, to name a few. Every phase of the process that led up to these
Critique: Assessment and the Production of Learning
759
completed movies was punctuated by episodes of critique in which the
young people reviewed one another’s works, discussing and debating
strengths, weaknesses, and possibilities for change.
One particularly heated instance of spontaneous critique arose during a
middle-of-the-night shoot, when Simon, a 17-year-old public high school
senior, was directing and starring in his own fictional movie about hostility
between two generations of Asian American youth. It was after midnight.
The group gathered in an urban alley, running extension cords out of
adjacent apartments, rigging lights, mics, and camera. Everyone in the
group was exhausted and frustrated. The day had started at 9:00 a.m., and
more than 15 hours later, it became apparent to the full group that Simon’s
fight scene was falling apart. It reached a point where Lila, the adult artist,
‘‘opted out,’’ as she said. She told the youth cast and crew that they were
on their own to figure out how to rework the scene and started packing
up the set.
Within this tense environment, it came time to shoot the pivotal moment
in Simon’s piece: when his character gets into a physical struggle with another character, played by Chanpory, at which point a gun accidentally fires,
killing Simon’s character’s brother (played by Simon’s real-life brother,
Bryant). While all this action was happening on camera, a great deal of work
was also taking place behind the scenes. Cleo scrutinized the action on a
field monitor, a portable television that allows video makers to see what
their footage looks like on screen. Claire operated the camera. Lydia, another member of the group, watched and waited. Claire called action.
Simon and Chanpory lunged toward each other and began the struggle,
with Claire rolling tape until Cleo stopped everything abruptly, saying ‘‘I’m
telling you, Simon, it’s not working. This is not working.’’ Simon at that
point was still focused on Chanpory, telling him exactly how to pull his arm
to make him fall, but Cleo was insistent, calling Simon’s name repeatedly
until he finally turned to her, even as Lydia, in the background, muttered
under her breath, ‘‘I feel like screaming.’’ Finally with Simon’s attention,
Cleo began to explain what she had seen. ‘‘All right, when I was looking at
it, it looked really weird, because—okay, what do you want the people to
see? You wanna see all four of you guys, or just two?’’ Simon and Chanpory
immediately began answering simultaneously, and even after Cleo told
Chanpory to let Simon talk (it was his movie after all), Chanpory continued
to direct the scene from his own point of view. ‘‘No! I don’t want to hear
from you!’’ she said, putting her hand up over Chanpory’s face, saying ‘‘I
don’t even want to hear from you.’’ Chanpory laughed and backed off,
letting a surprised-sounding, ‘‘Sorry!’’ pass through his lips. Simon then
explained his original vision of the scene, a mix of medium shots and wide
shots, from various angles. Cleo repeated his descriptions, making sure she
understood what he had in mind and prompting him to go on:9
760
Teachers College Record
1. Cleo:
2.
3.
4.
5.
6.
7.
Bryant:
Cleo:
Bryant:
Chanpory:
Cleo:
Bryant:
8. Chanpory:
9. Cleo:
Okay, then what? That’s okay. We’ll do that on the
close up, so then, this is what, when you guys are
fighting, you make sure Bryant is facing—
[Yeah, I did. I turned him around.
[these people, because I only saw your back.
You have to get up though.
Yeah, you guys have to find a way to get up.
And make it fast, okay?
All right, [after you get shot, lay down on the floor
away from the camera.
[Okay. When you come after me, focus on this hand,
and just put both hands—
And you guys, you guys, you guys, you’re not fighting. You’re acting. So let the other person take control, if that has to be, you know what I mean. Don’t be
struggling, cause that looks stupid.
QUALITIES OF INTERACTION: ACCESS TO JUDGMENT
A fleeting exchange such as this might easily be missed by observers looking
for obvious or tightly organized evaluation events (Lee, 2001). And yet this
kind of episode, as representative of a process that took various forms across
the production cycle of the Cutaway project, reveals how critique manifests
itself within moment-to-moment interaction, providing access to specific
learning opportunities by giving young people responsibility for developing
and applying judgment.10
First, this is an episode of joint assessment, meaning that young people
were turning to one another to see the work unfolding before their eyes in
new ways, evaluating its merits and pointing to its shortcomings. Critique is
often shot through with obvious evaluative terminology—what linguists call
assessment tokens—in this case, ‘‘It’s not working,’’ ‘‘It looked funny,’’ ‘‘That
looks stupid,’’ or even ‘‘I feel like screaming.’’ Youth in critique can also
convey assessment in less explicit ways—by silence, for example, or through
interpretations that imply respect for the work under review, or with the
unspoken ratification of uptake. In making these kinds of moves at this
juncture within Simon’s shoot, young people are constituting a circle of
decision makers at the precise moment when the adult in charge has strategically, if temporarily, vacated her position. The learning dynamic in place
here is fueled in part by the mutual understanding that ultimately the work
must be completed in a timely manner (before they are kicked out of the
alley), and that the finished product will be viewed and judged by a critical
outside audience.
Critique: Assessment and the Production of Learning
761
Particularly striking in the data both within Cutaway and across research
sites were instances in which young people dramatized narratives invoking
anticipated audience members, and other imaginary interlocutors, enacting
how those hypothetical figures would respond to the work. They conjured
characters and then voiced their own assessments as if they came through
that character’s mouth. In this excerpt, Cleo strategically alluded to what
Simon wanted ‘‘the people’’ to see, aligning her own questions with a
broader audience view. In this sense, she amplified her voice, recontextualizing the present decision—which was being made among peers
and collaborators in an urban alley—by projecting the consequences of that
decision onto an exhibition that would ultimately be assessed by a much
wider public. The young people themselves carried responsibility for judging what they wanted those people, that projected audience, to see. It must
be stated, however, that it was only after months of working together that
the adult artist could leave the young people to resolve this crisis; they had
developed habits of critique and a shared understanding of one another’s
projects, which allowed them to persevere and see both weaknesses of the
work and possibilities for salvaging Simon’s vision.
Second, critique entails moment-to-moment improvisation. Cleo, Simon,
and the others did not have the luxury of planning out in advance how to
manage this particular microcrisis of representation. With little time to
prepare what they would say, the young people improvised in the same way
that jazz musicians do on stage—by way of simultaneous composition and
performance, making ‘‘in-flight’’ decisions about problems without prefigured correct answers (Eisner, 1992; Thibault, 1997; see also Lee, 2001).
They could also immediately sense how interlocutors were receiving their
feedback and spontaneously adjust their commentary. Critique is therefore
not only iterative but responsive, evolving out of the specifics of a given
group within a particular context. Enormous planning went into Simon’s
movie, but not every detail was predictable. The young people necessarily
had to improvise their work on the movie, in large part by exchanging and
acting on impromptu assessments. This particular situation was a kind of
worst-case scenario—Simon’s carefully choreographed vision did not translate to the screen. Improvising one’s way through that kind of crisis without
an absolute authority who could come in and save the day meant calibrating
the critique to the dynamics of that precise moment. Never before in the
Cutaway process had someone summarily stopped someone else from talking (by inserting a hand in front of another person’s face) as Cleo did here.
But she apparently sensed that the urgency of the present situation necessitated that move, and the ensuing interaction appears to suggest that she
was not mistaken. Along with this example operating on the level of interaction, there were also technical skills that allowed these young people, in
this moment, to improvise their way through this instance of critique; their
762
Teachers College Record
mastery of vocabulary such as wide shots and medium shots allowed them
to speak in a kind of shorthand through which they could express, evaluate,
and finally execute the scene. Critique, in a broader sense, entails improvising a course of action out of the often conflicting positions circulating
within the group. Youth work toward a standard that emerges from their
collaboration with a professional and their developing knowledge of models
of ‘‘good work.’’
Reciprocity is a third defining feature of critique as an opportunity for
learning through the exercise of judgment. Within this moment during
Simon’s shoot, the young people exchanged perspectives, identified problems, and proposed solutions, not always harmoniously. Cleo edged into a
directorial role, even in a movie technically directed by one of her peers.
Once this episode passed, however, the dynamics of reciprocity shifted.
Later this same night, the group began shooting a scene from Cleo’s movie,
and soon they were critiquing her work. They energetically debated whether it would be clear that her cop character, who spouted antiyouth rhetoric,
was not expressing Cleo’s own views, but exposing the perspective that she
wanted to condemn.
Reciprocity is what distinguishes critique from one-way structures of
evaluation: Someone from outside an experience passes judgment on the
work without that person’s own efforts also being subjected to feedback
from the group. Reciprocity also differentiates critique from criticism, in the
literary or aesthetic sense; reviewers compose evaluations of completed
work without ever actually having to come face to face with author or artist,
who may or may not even still be alive. All participants in critique subject
work to joint review; there is a mutuality of engagement and vulnerability to
the judgment of others.
Fourth and finally, in critique there is an orientation toward emerging
work. Although participants in critique often (sometimes to a problematic
extent) look back toward the intentions that drive certain creative decisions,
they consider where the work comes from to determine where it needs to
go. Within this scene from Simon’s shoot, the young people repeatedly
projected a sense of future for the work and for their decisions even as they
entertained ‘‘past tense’’ considerations—including what Simon had originally envisioned for the piece and how specific actions registered on screen.
The youth projected what they could and could not do, forecasted how
certain moves would look, and asked questions to clarify planning for the
next take. By projecting an imaginary sense of future for the work, young
people in critique edge toward actual completion of a given project while
also developing a broader sense of how that which they have seen and
learned in one project might apply to new endeavors in related fields.
The imprint of critique can be seen on finished products through the
adjustments that authors have actually made on the basis of peer feedback,
Critique: Assessment and the Production of Learning
763
and evidence in some cases that authors chose not to bend to every judgment offered by colleagues and collaborators. In the case of Simon’s movie,
the change that came over the work as a result of critique was evident in the
contrast between the ways in which his peers initially reacted to his scenes,
with considerable negative feedback and frustration, and the response that
he received at a rough-cut screening, just days before the final exhibit,
attended by Cutaway youth and an adult professional editor consulting on
the project. When it came time to watch the fight scene and accidental
shooting, some in that audience were nearly speechless, others offered
emotional praise, and more than one person in that room actually teared
up, having been swept into the drama of Simon’s story playing out on the
screen.
SCENARIO 2: YOUTH AND ADULTS IN CRITIQUE AT YOUTH RADIO
The episode of critique drawn from Cutaway gives a picture of what
the process can look like and what learning opportunities can open up
when young people themselves manage the process of judgment. This
second example of critique shares the defining features identified above: It
is an instance of joint assessment, an improvisational, reciprocal process
oriented toward the future of the work under review. However, in this case,
the dynamics of critique include young people in a relationship with adults;
both parties depend on each other, in obvious and subtle ways, to complete
work that meets professional standards. Episodes of critique involving
young people and adults take place almost daily at Youth Radio, where
adult producers such as myself collaborate with youth reporters on stories
for local and national broadcast, most notably on National Public Radio.
Young people come into the program from San Francisco Bay Area public
schools, and 80% are working-class youth and youth of color—not the
voices typically dominating public airwaves, particularly as reporters, commentators, cultural critics, and analysts. An ethnically and economically
diverse staff brings diverse professional experiences to the work, including
backgrounds in broadcast journalism and music programming, youth
development, activism, and education.
The young people come to their first class at Youth Radio on a Wednesday, and by Friday, they are on the air for their live show in which they DJ
music segments, write and deliver commentaries, announce the news, and
produce public service announcements. Young people who are especially
interested in audio narrative pursue advanced training and eventually arrive in Youth Radio’s newsroom, the primary source of the organization’s
national stories. Youth Radio has aired national stories in the last few years
on topics including sexuality in middle school, standardized testing boycotts
764
Teachers College Record
in low-income districts, minors who volunteer as subjects in medical research, math requirements as a barrier to college retention, and commentaries from Palestinian and Israeli youth living through the crisis in the
Middle East. Youth reporters meet weekly in editorial meetings, where they
pitch story ideas and frame narratives around characters, ambient sound,
archived media, and music. The process of actually producing a story involves gathering tape, writing a detailed script, recording narration, digitizing these materials, and using editing software to mix the piece. Young
people carry out each of these phases with hands-on guidance from adult
producers. Peer critique is built into a place like Youth Radio on multiple
levels—in editorial meetings, for example, or when two people go out together to get tape for a story, or when one person is engineering another
person’s piece. There is also a longstanding model of peer teaching at Youth
Radio, where young people serve as editors, voice coaches, and studio
mentors to others just learning new skills. Within the apprenticing culture
of Youth Radio, these peer teachers are simply more experienced youth
who have been working for some time at Youth Radio, reaching the point at
which they are prepared to educate others; in some cases, their students are
older than they are.
Recently, Youth Radio solicited a poem from 17-year-old Rafael Santiago
Casal as part of a series that the newsroom was producing for public
radio on youth, money, and marketing. Rafael writes and performs
with a spoken-word poetry program called Youth Speaks, for which
I serve on the education committee of the board. Rafael delivers his poem
in a head-spinning rush of words and images, peppered with sexual
metaphors and profanities (which are, at least to this listener, neither
gratuitous nor predictable). This particular poem is a full-frontal attack on
the style industry, which he says uses child labor to commodify and sell
youth culture back to kids at a price that they cannot afford. Rafael starts
with a mocking reference to ‘‘the man with fashion sense’’ who had the
bright idea to tell kids what they need, creating ‘‘a million martyrs’’
identically dressed in sagged jeans, major league team jerseys, silver chains,
and $200 shoes. Later in the piece, Rafael moves from ‘‘the man’’ as the
target of his condemnation to a corporation he genders as a ‘‘damn fine’’
female who offers sex with just this catch: ‘‘you got to brand it and
lavish and ravage your own/image after we ho ya and handle your own
every dollar/until everyone marches to the same beat.’’ He links these media
manipulations to a kind of ‘‘mental incarceration’’ not unrelated to the
forces that ‘‘ended in the unfortunate/substantial transformation of our
pentagon into a quadrilateral.’’ And finally Rafael concludes, in this final
passage, with an image of his own body controlled by the manipulations he
describes—an ironic message, given the penetrating critique he has just
produced:
Critique: Assessment and the Production of Learning
765
From BET to MTV we got PYTs11 marketing sex appeal the
all mighty breast appeal
It sells, yes but reflects poorly on the women ya feel
Got us thinking woman? Oh you mean woo-man here to woo
the man from the man to the man just to do the man lie
down dick ride and screw the man please she fuckin
rules the man this is just the plan they planted
embedded empowering them through the image in women
and men til corporate America owns my dick, they tell
it where to be what to do and how high to go so
frequently that i don’t own it anymore I feel lied to
We move to their currents like a tide pool
A million martyrs marching to the same beat
What an eye full
Public radio listeners will know that this is not the kind of narrative most
often broadcast on those airwaves. Youth Radio producers were struck by
the power of the poet’s message and the intricacy and lyricism of his imagery. We also had to consider the broadcast standards we would face in
pitching the poem to our outlets’ editors, who sometimes traffic in more
straightforward reporting using established news formats. Through e-mail,
we shared our feedback with Rafael. He would need to consider shortening
the poem simply because of the slots allowed for ‘‘creative’’ segments such
as this one on a traditional public affairs show. Federal Communication
Commission guidelines prohibit cursing and graphic sexuality, and we suggested that there were some passages whose subtlety might be hard to
follow for radio listeners on their afternoon commute.
In response to this e-mailed critique and suggestions for revision, Rafael
basically declined to participate. Perhaps we had missed the message of the
poem, he said, which was about media manipulation of a personal truth.
Was that not what we were asking him to do by requesting significant edits?
The poet’s response set off an exchange of subsequent e-mails and face-toface conversations, whereby he and we figured out a way to move forward
with the pitch of his piece, given our editorial considerations and his
insistence on the integrity of the message.
766
Teachers College Record
This episode begins to suggest the relationships between youth, adults,
youth-serving institutions, and public cultures that enter and help shape
critique. What drew our interest was the content of Rafael’s poem, from a
vantage point of a listener, as a curator of youth voices. But Rafael, through
his response to our critique, brought us inside his message, as he says—
forcing reflexivity about the practice of critique, its functions and limits. The
learning that arose here—for Rafael, and for the others involved in this
exchange—inserts adults inside the learning opportunities made possible
through critique. Young people and adults are jointly and reciprocally assessing one another, through moment-to-moment improvisation, always
with an eye toward a future for the unfolding work. The adults involved—
their own habits, aims, and prospects—are very much embedded within
these conditions, in any context in which the logics of youth development
and professional production meet through the demands of mutual assessment. The version of the poem Rafael ultimately recorded for Youth Radio,
which eventually aired on a weekly public affairs show, provides evidence of
the impact of this episode of critique. In this case, however, unlike the
Cutaway example, it is the absence of change in the finished product that is
most striking and significant. The substance of the piece did not shift, and
only the profanities, which would have disqualified the poem from broadcast on any network, were eliminated from the artist’s original.12 However,
the learning that developed for the young people and adults who participated in these negotiations has lingered beyond this particular encounter,
informing subsequent moments when it has come time to make judgments
about work that might push listeners to listen, and think, and see their
worlds in new ways.
IMPLICATIONS FOR LEARNING PRACTICE: CONDITIONS
FOR CRITIQUE
Episodes of mutual assessment drawn from Cutaway and Youth Radio
provide a sense of the distinct learning opportunities made possible
through critique, as evident in the nuances of interaction and in the details
of finished products. However, this kind of analysis cannot sidestep an important set of questions: What guarantee do we have that critique will steer
youth down the right path? Specifically, would it not have been better to
have a professional choreographer on hand to monitor Simon’s fight scene
and simply place those bodies where they belonged? Is it not possible to
argue that part of the frustration everyone felt during that midnight shoot
was that everything at that moment was up for joint debate? Likewise, in the
case of Youth Radio, would it not it have been much more efficient had the
young people and adults involved in the critique surrounding Rafael’s
Critique: Assessment and the Production of Learning
767
poem simply taken that piece as it was from the outset, without getting into
the negotiations that in fact ensued? At its worst, cannot critique devolve
into something like a focus group, where every artistic decision passes
through a consensus-generating process that ultimately compromises idiosyncratic approaches and brave departures of imagination?13
The simple answer to these questions is yes. Critique can be misguiding,
whether young people are alone in critique or formulating judgments with
adults. The same can be said, of course, when adults are engaging in their
own joint creative pursuits. When scholars edit anthologies, or submit articles to refereed journals, or coteach classes, or plan conferences, it is always the case that joint assessment might ultimately steer individuals and
groups in ill-fated directions or yield disparate and contradictory recommendations. And yet as young people learn to negotiate episodes of joint
assessment, they develop habits and strategies for judging the quality of
their own and their peers’ work. They need practice in the exercise of
judgment if they are to realize that they need not rely exclusively on an
outside authority with the absolute power to evaluate their performances
even if ultimately, other people’s assessments matter deeply to them and
help determine the fate of their work.
There is also a second, and more complicated, answer to the question of
how we know that critique will actually have a positive rather than negative
effect on a given project or learning environment: to identify the conditions
associated with critique as a necessary practice rather than contrived activity.
Ironically, these conditions evoke and invert some of the principles most
commonly associated with standardized testing in schools. Four conditions
emerged from my analysis across research sites where critique does and
does not take hold. Comparison involved systematically reviewing transcripts and other records of interaction over the course of full production
cycles at the research sites to identify under what circumstances, within
specific programs and across different organizational models, critique tended to emerge among youth and between young people and adults. Critique
erupts as a resource for learning when:
1. STAKES ARE INTENSE
This observation is counterintuitive in the sense that we tend to position
nonschool activities, especially those based in the so-called expressive arts,
outside the academic subjects prioritized in school. English and math classes
are what really matter, the logic goes, and the beauty of the arts, and ‘‘extracurricular’’ environments in general, is precisely that they offer a kind of
oasis from the pressures that young people face in more rigorous school
subjects (Siegesmund, 2002; Soep, 2002). But in nonschool projects in
which young people choose to participate, the stakes are intense, and they
768
Teachers College Record
motivate critique precisely because the outcome of the project matters
enough to those involved that they want and need to judge their own efforts
and assess the work of their peers. Stakes create boundaries, limits, and
parameters. In Simon’s case, he knew that he had only one shot at capturing the scene he needed in that alley at midnight, given the group’s
incredibly tight production schedule and limited resources. It was serious
business, with a whole group of peers counting on his ability to come
through for this shoot, and with a finished piece alongside which their own
work would be displayed. Likewise, at Youth Radio, the nonnegotiable
deadlines associated with broadcasts added a sense of urgency to the critique surrounding Rafael’s poem. Very little mattered more to the young
people and the adults involved than their own sense of professional and
creative integrity and an ability to deliver the highest quality product to
professional editors and to the listeners who ultimately hear the stories on
the air. Researchers have used the term safe havens to denote effective arts
learning environments (Davis et al., 1993), and that term is apt, in a paradoxical sense, because it describes spaces that allow young people to experiment with the absence of safety inherent in the act of committing to a
project that actually matters to them and to a wider audience.
2. METASTANDARDS ARE SUBJECTED TO REVIEW
Critique takes root in places where criteria for judging the work under
review are themselves subjected to scrutiny. This observation is drawn out
by Fleetwood (2005) in her analysis of youth-based media arts organizations. Her analysis suggests that youth-adult collaboratives do not always
use critique to the fullest as a way to examine the tacit ideologies guiding
judgments about the work as it develops (see also Paley, 1995). When the
assumptions shaping production escape joint scrutiny, critique can only refine existing or accepted practices rather than unsettle those processes and
point to new and potentially more provocative directions for creative intellectual and political work. The capacity to expose and rework metastandards requires considerable sophistication. In this sense, a very basic
condition for critique is serious focus on skill development—which might
mean ‘‘old-school’’ lecture and demonstration formats alongside hands-on
experimentation—for novices within a given realm of practice to develop
the critical literacies they need to interrogate the basic assumptions and
‘‘received knowledge’’ driving work in their field.
3. ACCOUNTABILITY IS MUTUAL AND INTERACTIVELY SUSTAINED
Accountability is another buzzword in education, typically framed as a systematic process of evaluating whether an investment has paid off, and then
Critique: Assessment and the Production of Learning
769
assigning consequences—for example, determining if reallocations of a
school’s budget yield elevated test scores and making future funding decisions on that basis. But in environments that promote critique, accountability is a moment-to-moment affair. Certainly, young people in critique are
accountable to outside forces, and in fact, this specter of public release is
critical, as we have seen. They are also accountable to one another, often
quite literally—as is the case with Cutaway, where the young people act in
and ‘‘crew’’ for each other’s productions and where the movies need to link
thematically and aesthetically for the installation to work. In art forms such
as dance, a breakdown in peer accountability can have immediate physical
consequences if the person supporting another person’s body has not
trained sufficiently to carry the weight. Accountability can also play out in
the imaginative and ideological realm—within projects such as Youth Radio’s—in which there is an expectation that the final production or exhibition will have a specific impact, whether in the media world, in political
terms, or both. Young people negotiate the details of accountability within
moments of display—stopping, seeing, assessing, starting again—during
the course of production. All this occurs through the immediacy of
moment-to-moment interactions.
4. INTERDISCIPLINARITY IS MANDATORY
It is commonplace in education to think in terms of mastery within single
disciplines. Standardized test-driven learning environments foster this kind
of compartmentalization because paper-and-pencil tests lend themselves to,
and help produce, clearly circumscribed bodies of knowledge for measurement. But critique-rich environments very often combine multiple disciplines within collaborative projects. The interdisciplinary character of
Cutaway is obvious. It involves sound design, visual expression, performance, sculpture, and architecture. At Youth Radio, central to the critique
surrounding Rafael’s poem was a process of moving between expressive
genres, specifically spoken-word performance and broadcast journalism,
and the effort to find a compelling narrative mode that integrated aspects of
the two raised new questions that fueled the critique. Using a more expansive definition of disciplines, there are demands that extend across
symbol systems and fields of practice. At Cutaway, young people wrote
character and scene synopses, drew storyboards, talked through different
options, dramatized scenarios, viewed examples of work by established
artists, and so on. They also carried responsibility for tasks that we do not
normally associate with the arts as ‘‘pure’’ disciplines at all; for example,
they researched the legalities of taping minors without parental permission,
created strategies for obtaining additional equipment, prepared promotional materials, taught new skills to other young people, and so on. At
770
Teachers College Record
Youth Radio, young people’s participation in various roles within the organization, including making presentations to funders and calling meetings
to shape and respond to changes in agency policy, exposes them to leadership structures, internal communication practices, and organizational
learning that demand the exercise of judgment, and hence participation in
critique. Working alone and silently in a studio environment could not be
further from the reality of how learning operated in, and was integrated
within, these places where youth jointly produce exhibitions and performances for outside audiences.
Taken together, these conditions for critique inform efforts to engage
young people in assessment of their own work and that of their peers—a
process that has been linked with improved products and reflections—in
ways that are embedded within the demands of joint production.
IMPLICATIONS FOR LEARNING THEORY: LEARNING
AS PRODUCTION
These conditions for critique highlight the idea of production as both a
driving interactive goal and guiding conceptual metaphor through which to
theorize learning. Learning is not, in the episodes examined here, a set of
information or skills that young people acquire and simply carry with them
from one situation to the next. It is not, in other words, a thing to possess.
Moreover, learning cannot be reduced to an everyday, natural property of
ordinary life—a thing that simply happens in the course of any activity. But
learning is also more than a momentary or situated cognition accomplished
within a specific context. It is more, in other words, than a thing to do
(Chaiklin & Lave, 1996; Sfard, 1998). Learning is, in its manifestation
through critique, a way of making. Young people in critique are constantly
producing contexts for learning, producing situations in which they need to
act and judge. And they are constantly reorganizing their environments in
ways that extend their situated cognition beyond a given here and now—a
video shoot in an alley within a crisis of choreography, an e-mailed refusal to
make changes to a poem—into a future beyond that moment. In Simon’s
fight scene, as an example, moments of critique are steeped in aesthetic
concerns—lighting, narrative, conceits of realism, building of suspense
through sound effects and music, and so on. They are also driven by negotiations of authority, with the Cutaway group tacitly agreeing that Cleo
can stick her hand in Chanpory’s face and tell him, essentially, to shut up.
Learning as production means participating in countless fleeting moments
of judgment that fuel further action, engagement, and experimentation.
Critique asks learners to remake the situations in which they participate
by introducing imagined voices, times, and spaces—moving the interaction
Critique: Assessment and the Production of Learning
771
from its literal confines into hypothetical scenarios—a scene that is compelling to audiences or one that just ‘‘looks stupid,’’ to use Simon’s example.
The language of critique in particular is what linguists might call temporally
transcendent, to the extent that interlocutors ‘‘recontextualize’’ past experience and ‘‘precontextualize’’ future possibilities (Ochs & Jacoby, 1997)—a
way of feeding forward and not just feeding back. Whereas learning inhabits ‘‘communities of practice,’’ critique draws young people to produce
situations that transcend boundaries of voice, time, and space. This process
requires not distancing oneself from present activity by resorting to decontextualized or transferable bits of information (Chaiklin & Lave, 1996).
Rather, the process involves producing imaginary conditions and contingencies and projecting them beyond the immediate through moment-tomoment interaction.
IMPLICATIONS FOR RESEARCH ON LEARNING
These theoretical considerations carry implications for the production of
learning, but they also, by way of closing, contain new insights for the
production of research. Of particular relevance are instances of critique
involving young people working in collaboration with adults to produce
original work for distribution to real audiences in the hopes of having a real
impact. Young people and adults working along these lines offer a model
for scholars who want to work with youth as agents, and not only objects, of
research. Cultural studies scholars and education researchers increasingly
seek methods that move beyond the superficial use of youth for access,
quotation, consultation, and approval of findings (Fine, 1994; Heath,
1998a; Reason & Bradbury, 2001). In an article on the state of youth studies
as an interdisciplinary field, Cohen and Ainley (2000) highlighted new
methodological innovations—audio and video diaries, photo-mapping,
story-making—all used in an effort to find a space for meaningful, and
relatively reciprocal, scholar-youth collaborations (see also Lipsitz, 2001;
Tuhiwai Smith, 1999).
At places like Cutaway and Youth Radio, young media artists, working
with adults, observe and ask questions about environments relevant to their
stories. They record their surroundings and interactions using various
technologies, and then they develop imaginative ways to tell their stories to
an audience. These phases of work give a sense of new ways that scholars
might involve young people in the making of research as questioners, data
gatherers, analysts, and creative producers (Soep, 2003). Arts collaboratives, in which youth and adults jointly produce original narratives—video,
radio, poetry, and so on—are resources for researchers who initiate educational analysis together with young people.14 As Rafael reminded us,
772
Teachers College Record
creating a message, whether it fills the space of a museum gallery, broadcasts over audio airwaves, or materializes as an academic text, means conveying a significant truth. To do so requires addressing the possibilities of
manipulations operating on various levels, having the courage and resources to take them on, and figuring out a way to say something meaningful and multivocal. Innovative educational researchers have an
opportunity to bring young people’s epistemologies into conversation with
their own ideas and those drawn from published literature—using ‘‘youth
voices’’ not as raw materials to be interpreted but as already developed
theories to be reckoned, and sometimes wrestled, with—a process that fuels
further analysis. Such an approach requires a willingness on the part of
adults to subject ourselves to critique on the level of the stakes we assert, the
standards we hold dear, to whom and what we are accountable, and in what
breadth of roles we do and do not allow youth participation. Methodological projects carried out in the spirit of youth collaboration also have the
potential to socialize young people into professional fields of inquiry, moving well beyond token inclusion of youth as junior field workers or informants. This model for research is not possible without an ongoing
process of critique. And so this analysis of critique ends with a view of how
perhaps to begin again, with a revised approach to research that takes
critique not only as an object of study but also as a key dimension of the
process of investigation itself, involving young people and adults working
together to produce new understandings of learning.
Earlier versions of this article were delivered in two lectures: one as part of Harvard University’s John Landrum Bryant Lecture and Performance Series (2002) and one at the University of California at Berkeley’s program in Language and Literacy, Society and Culture
(2003). I would like to thank Shirley Brice Heath, Jessica Davis, Leisy Wyman, Rube´n
Gaztambide-Fernandez, and Ray McDermott for comments on drafts of this article.
Notes
1 It must be noted, however, that different contexts invite different modes of participation; for example, silence can itself serve as a potent response to a previously uttered assessment, and in this sense, not speaking can be its own form of participation.
2 Cutaway is a pseudonym, and I have also replaced the names of all individuals involved
in this organization. In the case of Youth Radio and Youth Speaks, however, with permission, I
use actual names.
3 Although college or university visual arts classrooms and studios are certainly not alone
as sites for peer critique, as Rubén Gaztambide-Fernandez has pointed out in his comments on
this article, critique should not be taken as an automatic feature of arts learning in general. One
need only think of the harrowing experiences that youngsters may have with a domineering
ballet or piano instructor. And in traditional theater settings, often the director controls the flow
of assessment through the delivery of ‘‘notes,’’ and actors are prohibited (at least officially)
Critique: Assessment and the Production of Learning
773
from commenting on one another’s efforts. Experimental theater often unsettles these conventions by engaging actors in both the performance and direction of emerging works.
4 Also relevant are studies of a practice similar to arts critique in writing pedagogy—the
‘‘workshop,’’ where students review one another’s essays. See, for example, Cazden, 1996;
Delpit, 1986; DiPardo & Freedman, 1988; Gutierrez, 1992; Lensmire, 1998; Leverenz, 1994;
Reyes, 1992; Sperling, 1996.
5 For the following discussion of three ways to conceive of learning, I am indebted to Ray
McDermott (personal communication, 2001).
6 That research investigated youth organizations primarily in working-class neighborhoods within urban centers, midsized towns, and rural areas—places ranging from tumbling
teams and midnight basketball leagues to agriculture organizations, improv theater groups,
and peer tutoring centers (Heath & McLaughlin, 1987, 1993, 1994a, 1994b).
7 Site selection for my in-depth research was based on five criteria: (1) these sites operated
outside of schools, thereby illuminating learning beyond traditional classrooms; (2) they centered on the arts and thus featured projects that involved the negotiation of personal and
collective meanings and the risk of disclosure; (3) they focused on sustained projects in which
critique extended through periods of planning, preparation, practice, and performance—
cycles associated with effective youth-based learning environments (Heath & McLaughlin,
1993); (4) they were tuition free and involved ethnically and economically diverse groups of
youth and instructors who availed themselves to the generation of learning theory based on the
lives of young people who bring a complex range of experiences to the work at hand and do
not necessarily conform to habits of speech or development that mark learning environments
comprising White middle-class learners and teachers; and (5) participants at these sites
expressed interest in my observing, and in some cases joining, their activities.
8 Cutaway students were Cleo (Chinese American), Simon (Chinese American), Jason
(Filipino American), Alison (Caucasian), Jamal (African American), Cassie (Chinese American/
Caucasian/Latino), Claire (Korean American), and Lydia (Vietnamese American). Chanpory
and Bryant, two friends who acted in one of the movies that Cutaway produced, were both
Asian/Pacific Islander. With the exception of Claire and Alison, all attended urban public or
parochial schools.
9 In this transcript, brackets mark the beginning of a passage where two young people
speak simultaneously.
10 Along with these responsibilities for judgment come experiences with collaborative
leadership, as documented by Roach and her colleagues (1999).
11 BET is Black Entertainment Television; MTV is Music Television; PYTs are ‘‘Pretty
Young Things.’’
12 This does not, of course, mean that young people’s ‘‘raw’’ voices should be left as is,
without the benefit of thoughtful response from peers and adults, only that these kinds of
decisions need to be made on a case-by-case basis and that even a final outcome with few
changes to the finished work can sometimes mask a more complex process of negotiation.
13 For this important line of questioning, I am indebted to Elliot Eisner (personal communication, 2000).
14 These sorts of projects also have relevance to proposals in the literature regarding
artistically grounded approaches to research (see, for example, Eisner, 1995; Lightfoot & Davis,
1998).
References
Anderson, J., Reder, L., & Simon, H. (2000). Applications and misapplications of cognitive
psychology to mathematics instruction. Texas Educational Review, 1(2), 29–49.
774
Teachers College Record
Barrett, T. (1988). A comparison of the goals of studio professors conducting critiques and art
education goals for teaching criticism. Studies in Art Education, 30, 22–27.
Barrett, T. (1994). Principles for interpreting art. Art Education, 47(5), 8–13.
Barrett, T. (1997). Talking about student art. Worcester, MA: Davis.
Barrett, T. (2000). Studio critiques of student art: As they are, as they could be with mentoring.
Theory Into Practice, 39, 29–35.
Blythe, T., Allen, D., & Schieffelin Powell, B. (1999). Looking together at student work: A companion
guide to assessing student learning. New York: Teachers College Press.
Bulka, J. (1996). You call that art? A critique of the critique. The New Art Examiner, 23(6), 22–25.
Campbell, S. (2001). Shouts in the dark: Community arts organizations for students in rural
schools with ‘‘urban’’ problems. Education and Urban Society, 33, 445–456.
Cazden, C. (1996). Selective traditions: Readings of Vygotsky in writing pedagogy. In D. Hicks
(Ed.), Discourse, learning, and schooling (pp. 165–188). New York: Cambridge University
Press.
Cazden, C. (2001). Classroom discourse: The language of teaching and learning. Portsmouth, NH:
Heinemann.
Chaiklin, S., & Lave, J. (1996). Understanding practice: perspectives on activity and context. Cambridge, UK: Cambridge University Press.
Cohen, P., & Ainley, P. (2000). In the country of the blind? Youth studies and cultural studies in
Britain. Journal of Youth Studies, 3, 79–95.
Cohen, E., Lotan, R., Scarloss, B., Schultz, S., & Abram, P. (2002). Can groups learn? Teachers
College Record, 104, 1045–1068.
Cotton, C. (1981). Critique: A middle school evaluation system. School Arts, 80(5), 46–47.
Darling-Hammond, L., Ancess, J., & Falk, B. (1995). Authentic assessment in action: Studies of
schools and students at work. New York: Teachers College Press.
Davis, J. (1998). Everything old is new again: Self-assessment as tradition in community art
centers. In R. E. Stake, K. G. Congdon, & D. Boughton (Eds.), Advances in program evaluation: Volume 4. Evaluating art education programs in community centers: International perspectives
on problems of conception and practice (pp. 117–132). Stamford, CT: JAI Press.
Davis, J., Soep, E., Maira, S., Remba, N., & Putnoi, D. (1993). Safe havens: Educational effectiveness at community art centers that focus on education in economically disadvantaged neighborhoods.
Cambridge, MA: Harvard Project Zero.
Delandshere, G. (2002). Assessment as inquiry. Teachers College Record, 104, 1461–1484.
Delpit, L. (1986). Skills and other dilemmas of a progressive black educator. Harvard Educational Review, 56, 379–385.
DiPardo, A., & Freedman, S. (1988). Peer response groups in the writing classroom: Theoretical foundations and new directions. Review of Educational Research, 58, 119–147.
Eisner, E. (1992). The misunderstood role of the arts in human development. Phi Delta Kappan,
73(8), 591–595.
Eisner, E. W. (1995). What artistically crafted research can help us to understand about schools.
Educational Theory, 45, 1–7.
Ende-Saxe, S. (1990). The elementary critique. School Arts, 90(3), 22–23.
Filer, A. (2000). Assessment: Social practice and social product. London: RoutledgeFalmer Press.
Fine, M. (1994). Dis-stance and other stances: Negotiations of power inside feminist research.
In A. Gitlin (Ed.), Power and method: Political activism and educational research (pp. 13–35). New
York: Routledge.
Fleetwood, N. (2005). Authenticating practices: Producing ‘‘the real’’ in youth videos. In
S. Maira & E. Soep (Eds.), Youthscapes: The Popular, the National, the Global. Philadelphia:
University of Pennsylvania Press, 155–172.
Fox, R. (2001). Constructivism examined. Oxford Review of Education, 27, 23–34.
Critique: Assessment and the Production of Learning
775
Goodwin, C., & Goodwin, M. H. (1992). Assessments and the construction of context. In
A. Duranti & C. Goodwin (Eds.), Rethinking context: language as an interactive phenomenon
(pp. 147–190). New York: Cambridge University Press.
Greeno, J. (1998). The situativity of knowing, learning and research. American Psychologist,
53(1), 5–26.
Gutierrez, K. (1992). A comparison of instructional contexts in writing process classrooms with
Latino children. Education and Urban Society, 24, 244–262.
Hacking, I. (2000). The social construction of what? Cambridge, MA: Harvard University Press.
Heath, S. B. (1993). Inner-city life through drama: Imagining the language classroom. TESOL
Quarterly, 27, 177–192.
Heath, S. B. (1998a). The arts in the nonschool hours: Strategic opportunities for meeting the
educational, civic learning, and job training goals of America’s youth. Research report
presented to the President’s Committee on the Arts and Humanities, March 2. Washington,
DC.
Heath, S. B. (1998b). Working through language. In S. Hoyle & C. T. Adger (Eds.), Kids talk:
Strategic language use in later childhood (pp. 217–240). New York: Oxford University Press.
Heath, S. B. (2001). Three’s not a crowd: plans, roles, and focus in the arts. Educational Researcher, 30(7), 10–17.
Heath, S. B., & Ball, A. (1993). Dances of identity: Finding an ethnic self in the arts. In S. B.
Heath & M. McLaughlin (Eds.), Identity and inner city youth: Beyond ethnicity and gender (pp.
69–93). New York: Teachers College Press.
Heath, S. B., & McLaughlin, M. (1987). A child resource policy: Moving beyond dependence
on school and family. Phi Delta Kappan, 68, 576–580.
Heath, S. B., & McLaughlin, M. (1993). Identity and inner city youth: Beyond ethnicity and gender.
New York: Teachers College Press.
Heath, S. B., & McLaughlin, M. (1994a). Learning for anything everyday. Journal of Curriculum
Studies, 26, 549–567.
Heath, S. B., & McLaughlin, M. (1994b). The best of both worlds: Connecting schools and
community youth organizations for all-day, all-year learning. Educational Administration
Quarterly, 30, 278–300.
Heath, S. B., & Soep, E. (1998). Youth development and the arts in the non-school hours.
Grantmakers for the Arts, 9(1), 9–32.
Hogan, K. (2002). Pitfalls of community-based learning: How power dynamics limit adolescents’ trajectories of growth and participation. Teachers College Record, 104, 586–624.
House, N. (2001). Critiques in the K-12 classroom. NAEA Advisory. http://www.nea-reston.org/
pdf/2crit.pdf. Retrieved January 27, 2006.
Hull, G., & Schultz, K. (2001). Literacy and learning out of school: A review of theory and
research. Review of Educational Research, 71, 575–611.
Hull, G., & Schultz, K. (2002). School’s out: Bridging out-of-school literacies with classroom practice.
New York: Teachers College Press.
James, P. (1996). The construction of learning and teaching in a sculpture studio class. Studies in
Arts Education, 37(3), 145–159.
Lave, J., & Wenger, E. (1991). Situated learning: legitimate peripheral participation. Cambridge, UK:
Cambridge University Press.
Lee, C. (2001). Signifying in the zone of proximal development. In C. Lee & P. Smagorinsky
(Eds.), Vygotskian perspectives on literacy research: Constructing meaning through collaborative inquiry (pp. 191–225). Cambridge, UK: Cambridge University Press.
Lensmire, T. (1998). Rewriting student voice. Journal of Curriculum Studies, 30, 251–291.
Leverenz, C. S. (1994). Peer response in the multicultural composition classroom: Disensus—a
dream (deferred). Journal of Advanced Composition, 14, 167–186.
Lightfoot, S. L., & Davis, J. (1998). The art and science of portraiture. San Francisco: Jossey-Bass.
776
Teachers College Record
Linde, C. (1997). Evaluation as linguistic structure and social practice. In B. L. Gunnarsson,
P. Linnell, & B. Nordberg (Eds.), The construction of professional discourse (pp. 151–172).
London: Longman.
Lipsitz, G. (2001). American studies in a moment of danger. Minneapolis: University of Minnesota
Press.
Mahiri, J. (2003). What they don’t learn in school: Literacy in the lives of urban youth. New York: Peter
Lang.
Noam, G., Biancarosa, G., & Dechausay, N. (2002). Afterschool education: Approaches to an emerging field. Cambridge, MA: Harvard Education Press.
Ochs, E., & Jacoby, S. (1997). Down to the wire: The cultural clock of physicists and the
discourse of consensus. Language in Society, 26, 479–505.
Paley, N. (1995). Finding art’s place: Experiments in contemporary education and culture. New York:
Routledge.
Papert, S. (1991). Situating constructionism. In S. Papert & I. Harel (Eds.), Constructionism (pp.
1–12). Norwood, NJ: Ablex Publishing Company.
Reason, P., & Bradbury, H. (2001). Handbook of action research. Thousand Oaks, CA: Sage.
Reyes, M. (1992). Challenge venerable assumptions: Literacy instruction for linguistically different students. Harvard Educational Review, 62, 427–446.
Roach, A., Wyman, L., Brookes, H., Chavez, C., Heath, S. B., & Valdes, G. 1999. Leadership
giftedness: Models revisited. Gifted Child Quarterly, 43(1), 13–24.
Rogers, J. (1996). The concept of framing and its role in teacher-student negotiations during desk
critiques in the architectural design studio (Unpublished doctoral dissertation, Texas A & M
University, College Station, TX.
Rogoff, B. (1994). Developing understanding of the idea of communities of learners. In Mind,
Culture, and Activity, 1(4), 209–229.
Roth, R. (1999). The crit. Art Journal, 58, 32–35.
Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27(2), 4–13.
Siegesmund, R. (2002). Bringing accountability to elementary art. Kappa Delta Pi Record, 39,
24–28.
Soep, E. (1996). An art in itself: Youth development through critique. New Designs for Youth
Development, 12(4), 42–46.
Soep, E. (2000). To make things with words: Critique and the production of learning (Unpublished
doctoral dissertation, Stanford University, Stanford, CA.
Soep, E. (2002). Abnormal educational vision: Art in the city beyond school. Kappa Delta Pi
Record, 39, 12–16.
Soep, E. (2003). Learning about research from youth media artists. Penn GSE Perspectives on
Urban Education, 2(1). http://www.urbanedjournal.org/archive/issue3/commentaries/com
ment0004.html. Retrieved January 27, 2006.
Soep, E. (2004). Visualizing judgment: Self and peer assessment in visual arts education. In
E. Eisner & M. Day (Eds.), Handbook of research and policy in the visual arts.
Soep, E. (2005). Making hard-core masculinity: Teenage boys playing house. In
S. Maira & E. Soep (Eds.), Youthscapes: The popular, the national, the global (pp. 667–687).
Mahwah, NJ: Lawrence Erlbaum Publishers.
Sperling, M. (1996). Revisiting the writing-speaking connection: Challenges for research on
writing and writing instruction. Review of Educational Research, 66, 53–86.
Tannock, S. (1998). Noisy talk: Conversation and collaboration in a youth writing group. In
S. Hoyle & C. T. Adger (Eds.), Kids talk: Strategic language use in later childhood (pp. 241–266).
New York: Oxford University Press.
Thibault, M. (1997). Personal Communication.
Critique: Assessment and the Production of Learning
777
Trend, D. (1997). The fine art of teaching. In H. Giroux & P. Shannon (Eds.), Education and
cultural studies: Toward a performative practice (pp. 249–258). New York: Routledge.
Tuhiwai Smith, L. (1999). Decolonizing methodologies: Research and indigenous peoples. London: Zed
Books.
Varenne, H., & McDermott, R. (1999). Successful failure: The school America builds. Boulder, CO:
Westview Press.
Weiss, L., & Fine, M. (2000). Construction sites: Excavating race, class, and gender among urban youth.
New York: Teachers College Press.
Wernik, U. (1985). Psychological aspects of criticism in an academy of art and design. Journal of
Creative Behavior, 19, 194–201.
Young, M. F. D. (2000). Rescuing the sociology of educational knowledge from the extremes of
voice discourse: Towards a new theoretical basis for the sociology of curriculum. British
Journal of Sociology of Education, 21, 523–535.
ELISABETH SOEP received her PhD from Stanford University and has
taught at University of California, Berkeley, and San Francisco State
University. Her research, featured in national and international journals,
centers on youth learning, language, and cultural production in non-school
settings. Dr. Soep recently coedited Youthscapes: The Popular, the National, the
Global (with S. Maira, 2005, University of Pennsylvania Press). She is
currently working on a new book, Making the News: Youth Radio, Education,
and Media Culture (with V. Chávez, University of California Press). Soep is
the education director and a senior producer at Youth Radio, where she
collaborates with young media artists on stories for local and national
outlets, including National Public Radio. Youth Radio has been recently
recognized with honors, including the George Foster Peabody and Edward
R. Murrow Awards.