Schank, SRI International
NSF Grant IIS-0713711
ScribbleProv: Supporting Disciplined Improvisation During Face-to-Face
Discussion
FINAL REPORT: ACTIVITIES
Below we summarize project activities conducted during the final 6 months of the
ScribbleProv project, including completion the analysis of our classroom implementations,
collaborations with international colleagues, and dissemination of our work. In summary:
• Group Scribbles, along with interactive activities and pedagogical patterns, was
implemented in classrooms of five sixth-grade earth science teachers in Colorado.
• Observers conducted classroom observations and surveyed the teachers to obtain a
comprehensive view of all the activities and patterns they implemented.
• We worked closely with international colleagues in Singapore, Greece, and Spain to
improve the Group Scribbles code and identify future collaboration opportunities.
• Two conference papers were submitted to CSCL 2011.
Classroom Implementation Study
During the 2009-2010 school year, Group Scribbles patterns and activities were tested with
five sixth-grade Earth science teachers in Denver, Colorado1. The five teachers had experience
implementing a widely used Earth systems science curriculum, Investigating Earth Systems
(IES), developed by the American Geological Institute. The Group Scribbles activities were
integrated into the IES curriculum and specified questions for teachers to pose; how classroom
network technology will be used to support collection, aggregation, and display of data; and how
teachers can use assessment information to organize instruction. Pedagogical patterns successful
in prior research for promoting individual and group learning were used as templates for
designing the activities (DeBarger et al., 2010).
The five sixth-grade teachers came together for the first time in a professional development
workshop in August 2009 that introduced the teachers to Group Scribbles technology and
pedagogical patterns. The teachers then worked together in small groups with a science
assessment researcher, curriculum developer, and subject matter expert to develop activities.
After the workshop, the project team developed additional activities and worked with the five
teachers to pilot-test the activities in their classrooms in Colorado. Each teacher taught at least
three classroom sections in Earth science with 25-35 students each. The team engaged the
teachers in regular (approximately monthly) 1.5-hour teleconferences in which (1) technology
issues that teachers encountered were addressed, (2) teachers reported on an activity that they
implemented with students and described how it went, and (3) the team shared tips related to
effective use of the technology. Technological support was also provided through web
conferencing technology.
During the school year, we observed 12 classroom sections where the teachers used Group
Scribbles activities. The “Group Data Creation and Comparison” pattern was observed most
often, in half of the classrooms (6 observations), followed by “Where on This Image” (4
1
Our original plan had been to complete classroom observations in fall 2009, but due to the time it took us to reach
agreement with Denver Public Schools around the technology (e.g., teacher laptops) to be used, the teachers did
not have an adequate opportunity to complete their use of the Group Scribbles activities and technology until
spring 2010.
1
Schank, SRI International
NSF Grant IIS-0713711
observations), “Predict with Reasons” (1 observation) and “Question Posing and Categorization”
(1 observation). (See the Year 3 report for descriptions of these patterns.) Using semi-structured
observation protocols, observers recorded the focal science topic and described teachers’ and
students’ interactions. At the end of the class, observers recorded summaries of the class,
including instructions teachers gave, teacher support for student engagement, variations in
students’ responses to the activity, questions posed and nature of the responses, breakdowns in
the flow and management of activities, technology use, and communication of science content by
the teacher. At the end of the year, an online survey was fielded to obtain a comprehensive view
of all the activities and patterns the teachers attempted to implement and to identify activities and
patterns that were more and less valuable or usable from the teacher’s perspective. Results and
observations from classroom testing are summarized in the Findings section of this report.
Synergistic Activities
Group Scribbles continues to being actively used by researchers and students around the
world. During the final 6 months of the project, we worked closely with international colleagues
in Singapore, Greece, and Spain to improve the Group Scribbles codebase, prepare joint
conference submissions, and identify future collaboration opportunities.
National Institute of Education, Singapore
Dr. Chee-Kit Looi and colleagues at the National Instutute of Education in Singapore have
introduced Group Scribbles to more than 10 teachers and 200 students over the past 3 years,
working with the teachers in science, math, English, and Chinese language learning. Toward the
goals of supporting Group-Scribbles-based teaching and learning innovation, they have
identified nine principles to support rapid collaborative knowledge improvement (RCKI) in the
design of lessons, and worked with teachers to co-design lesson plans and apply these principles
(Looi, Chen & Ng, 2010). In the past year, Singapore also rewrote the backend of the Group
Scribbles system to fix issues with the Flex platform that resulting in dropped messages that
degraded interaction as they attempted to scale usage. In September 2011, Singapore contributed
these improvements back to SRI, along with new features including the ability to embed images
in scribbles, display video in boards, and import and export activities for sharing in a web-based
Group Scribbles activity library. SRI is calling the improved codebase Group Scribbles 3.0 and is
disseminating it to international colleagues through a shared source code repository.
University of Piraeus, Greece
Dr. Symeon Retalis, professor in the Department of Technology Education at the University
of Piraeus in Greece, visited SRI on sabbatical for two months during the summer of 2011 to
work with our project team on issues around learning design and Group Scribbles. During his
visit, we shared our research and findings related to the use of pedagogical patterns in the
classroom and supporting innovation in CSCL. As a result of our collaborations, Dr. Retalis is
incorporating use of Group Scribbles 3.0 into his idSpace project (http://www.idspaceproject.org) on collaborative innovation and we are collaborating on a joint workshop
submission to CSCL 2011.
University of Valladolid, Spain
Graduate students Luis Prieto and Sara Villagrá won fellowships from the University of
2
Schank, SRI International
NSF Grant IIS-0713711
Valladolid in Spain to visit SRI as International Fellows for three months during the fall of 2011.
During their visit, we shared and discussed results of Group Scribbles research across our two
projects, co-authored two submissions to CSCL (see below) that synthesized findings across our
projects, and held a workshop to reflect on ways to encourage teachers’ capacity to innovate with
Group Scribbles in the classroom. In November, Mr. Prieto internationalized Group Scribbles 3.0
and localized it to Spanish. Building on Mr. Prieto’s contributions, colleagues in Greece,
Finland, and France are planning to localize the new codebase to their languages and contribute
their localization files back to SRI for further distribution.
Dissemination
In the last 6 months year of the project, we collaborated with colleagues in Spain and
Singapore (mentioned above) to create two submissions to the CSCL 2011 conference.
The first paper describes our two independent research projects that aim to influence
authentic practices to support improvisatory adaptation of teaching using Group Scribbles.
Evidence from both projects shows that merely providing technologies that support this
adaptation is not enough to provoke the change and that exposing teachers to good uses of the
tool (in the form of pedagogical patterns) also has limited success. Both projects highlight the
difficulty for practitioners to bridge the gap between de-contextualized advice and contextualized
classroom situations. We propose the use of more atomic, actionable moves to help teachers
orchestrate technology to support deeper collaborative knowledge building.
Prieto, L., Schank, P., Villagrá, S., Penuel, W., & Dimitriadis, Y. (2010). Mind the
gaps: Using patterns to change everyday classroom practice towards contingent
CSCL teaching. Manuscript submitted to the 2011 International Conference on
Computer Supported Collaborative Learning.
The second paper proposes a symposium expand the notion of CSCL to understand how
people can become more innovative. The Group Scribbles teams from SRI and Spain will lead
one of the 4 proposed presentations, focusing on innovation through attention to successful
patterns of use. We focus on tools to help teachers innovate while using technologies like Group
Scribbles by helping them build a repertoire of teaching strategies to select from and adapt as
they make ongoing, creative adjustments to their practice, especially in reaction to a variety of
rich, constructed student responses. The other sessions focus on innovation while scaling Group
Scribbles in Singapore (Looi, Chen, Chan), CSCL and innovation among school leaders (Patton,
Chan), and innovation and CSCL for schools of education (Roschelle and Patton).
Roschelle, J., Patton, C., Schank, P., Penuel, W., Looi, C-K, Chen, W., Chan, A.,
Prieto, P, Villagrá, S., & Dimitriadis, Y. (2010). CSCL and innovation: In
classrooms, with teachers, among school leaders, in schools of education. Manuscript
submitted to the 2011 International Conference on Computer Supported Collaborative
Learning.
Our Singapore colleagues also published a journal article on their use of Group Scribbles in
Singapore classrooms to support rapid collaborative knowledge improvement (RCKI), and our
book chapter on Group Scribbles moved from in press to published by IGI Global.
3
Schank, SRI International
NSF Grant IIS-0713711
Looi, C.K., Chen, W. & Ng, F-K. (2010). Collaborative activities enabled by Group
Scribbles (GS): An exploratory study of learning effectiveness. Computers &
Education, 54 (1), 14-26.
DeBarger, A., Penuel, W. R., Harris, C. J., & Schank, P. (2010). Teaching routines to
enhance collaboration using classroom network technology. In F. Pozzi & D. Persico
(Eds.), Techniques for fostering collaboration in online learning communities:
Theoretical and practical perspectives (pp. 224-244). Hershey, PA: IGI Global.
4
Schank, SRI International
NSF Grant IIS-0713711
ScribbleProv: Supporting Disciplined Improvisation During Face-to-Face
Discussion
FINAL REPORT: FINDINGS
In this final report, we document findings from our classroom implementation study, and
discuss the implications of these findings and how they align with related research. In summary:
• Teachers enacted most of the provided GS activities and patterns, also created several of
their own GS activities based on the provided patterns.
• However, teachers experienced many tensions around classroom management, such as
technical issues sidetracking lesson flow, figuring out the “right amount of time” to allow
students to answer questions, and keeping students on task during group work.
• The quality of the student participation, and the contingent teaching observed, were
limited. Teachers often asked students to explain their ideas, but the teachers did most of
the intellectual work of building on and connecting ideas and rarely actively engaged
students in discussion one another’s ideas.
• We concluded that teachers need a broader suite of tools to improve the quality of
enactment of patterns in classrooms. Technologies designed for collaborative discussion,
even when complemented with patterns designed to provide opportunities for students to
share their thinking with others, do not necessarily yield rich discussion; a dialogic style
is a critical component of teaching that enables student agency and productive
collaborations.
• We propose that complementing a focus on patterns with the use of more atomic,
actionable moves (e.g., in the form of classroom norms, discussion moves, and decision
rules) will help teachers better orchestrate technology to support deeper collaborative
knowledge building.
Classroom Implementation Study
Use of Group Scribbles patterns and activities
Teacher surveys and classroom observations indicated that the five Denver teachers could
enact the Group Scribbles activities and patterns and that they did so many times during the 5
months when they were using the IES units. On average, each teacher used Group Scribbles
lessons six times in their classroom during the 2009-2010 school year. Table 1 shows a count of
pedagogical pattern use as reported by our Denver teachers on the end-of-year survey. The two
most widely used patterns were “Where on This Image?” And “Group Data Creation and
Comparison.” (See the Year 3 report for descriptions of these patterns.)
The teachers reported that they implemented most, but not all, of the Earth science activities
(see Tables 2 and 3); they also invented several of their Group Scribbles activities, primarily
based on the “Where on this Image?” and “Predict with Reasons” patterns.
1
Schank, SRI International
NSF Grant IIS-0713711
Table 1. Teachers’ use of pedagogical patterns.
Group Scribbles Pattern
Number of Reported
Uses
Where on this Image?
Group Data Creation and
Comparison
Predict with Reasons
Question Posing and Categorizing
Concept Mapping
Design and Test
9
9
3
4
3
2
Table 2. Number of teachers who used Rocks and Landforms activities.
Group Scribbles Activity
Investigation 1: Researching Rock Types
Investigation 2: Locating Places on a Contour Map
Investigation 3: Devising Models of Rock Abrasion
Investigation 4: Explaining Rock Abrasion
Investigation 5: Erosional Landforms
Investigation 6: Deposition
Investigation 7: Deltas and Floodplains
Investigation 8: Glacial Erosion and Deposition
Number of Teachers
Who Used Activity
2
4
0
1
2
2
0
3
Table 3. Number of teachers who used Dynamic Planet activities.
Group Scribbles Activity
Investigation 1: Mapping Questions to Types of Models
Investigation 2: Observing Waves & Measuring Wave Speed
Investigation 3: When Lithospheric Plates Meet
Investigation 4: Plate Boundaries
Investigation 5: Earthquakes, Volcanoes and Mountains
Investigation 6: Pangaea Activity
Investigation 7: Fast and Slow Changes to Earth’s Spheres
from Volcanoes
Number of Teachers
Who Used Activity
5
3
3
2
2
3
0
2
Schank, SRI International
NSF Grant IIS-0713711
Classroom management, classroom discussion, and contingent teaching
Survey results and classroom observations suggested promising levels of adoption, but they
also highlighted challenges to implementation and the need for a broader set of tools to improve
the quality of enactment of patterns in the classroom.
Initially, procedural aspects of the technology overwhelmed teaching and learning goals.
For example, setting up Group Scribbles for classroom use and preparing students for use at the
beginning of lessons had an impact on instructional time and teachers’ instructional focus. We
observed several early Group Scribbles lessons in which the procedural aspects, such as getting
all students up and running (and keeping students on pace) became the central focus for teachers.
Even as the procedural aspects became routinized over time, teachers still experienced tensions
in classroom management such as technical issues sidetracking lesson flow, figuring out the
“right amount of time” to allow students to answer questions, and keeping students on task
during group work.
“I want to have more training activities for my students to learn about it
before they have to use it. I felt that was the hardest part this year; they
were so enthralled to find out things that it was tough to manage.” [Teacher
teleconference, 2010/1/21]
“The only issue for me is when they're doing the wrong things—writing on
each other's boards, or writing a scribble that says "hi john". I take away
their computer after a warning to stop doing that.” [Teacher teleconference,
2010/3/25]
As teachers became more comfortable with the technology, especially how to set up in class,
get students started, and address glitches during lessons, we began to see a shift from primarily
technology troubleshooting by teachers to a focus on student thinking. The teachers reported that
the patterns advanced the goals of enhancing communication, motivation, and feedback and that
the activities helped students learn high-level skills. For example, after enacting a Group
Scribbles investigation of Pangaea, a teacher wrote:
“During this lesson, I think Group Scribbles worked very well. The presence of
computers took student engagement to a level that pencil and paper cannot... I
learned that students have misconceptions about how long ago Pangaea existed,
that they have trouble wrapping their minds around that immense length of time,
and that many of them do have a lot of background knowledge about Pangaea.
The activity was also beneficial because I was able to see that students struggled
with what ‘evidence’ really is.” [Classroom observation, 2010/1/12]
Still, the quality of the student participation and the contingent teaching observed were
limited. Discourse with Group Scribbles tended to focus on explaining evidence and claims
posted on group boards and included such prompts as “What do you notice about the data?”,
“What is the evidence you have for this?” and “Tell us why you think yes or no.” Observers
noted that teachers often asked student to explain their ideas, but teachers did most of the
intellectual work of building on and connecting ideas and rarely actively engaged students in
3
Schank, SRI International
NSF Grant IIS-0713711
discussion of one another’s ideas. Instead, teachers would address issues by restating the correct
ideas or explaining why certain answers were incorrect. On one occasion, observers saw a
teacher try several times to explain a scientific idea related to divergent plate boundaries. After
several re-explanations, she asked students to “show by giving thumbs up, to the side, or down”
the level of understanding they felt they had achieved regarding the explanation. For those who
displayed a thumb down (still confused) or to the side (not completely sure), the teacher told
them that they could return to the classroom during lunch for further review.
Finally, although teachers often highlighted key features of students’ responses to help all
students understand important ideas, there was little evidence of teachers changing the direction
of lessons to address clearly problematic ideas. Across observed lessons, teachers tended to stay
on the intended lesson path. The research team concluded that teachers needed a broader suite of
tools to improve the quality of enactment of patterns in classrooms.
Based on these findings, we see a need to support a more dialogic style in the enactment
of Group Scribbles patterns and activities. A dialogic style, in which teachers attempt to respond
to a student’s contribution from the student’s perspective and use their response to invite
additional student responses, is a critical component of teaching that enables student agency and
productive collaborations (O'Connor & Michaels, 2007). In response to this perceived need, a set
of classroom norms for participation, discourse moves to support richer classroom discussion,
and decision rules to provide contingency guidance to teachers were developed and are being
implemented with 15 Denver teachers under a separate project (Contingent Pedagogies). Norms
set expectations about how students will participate in discussion and establish a classroom
community with a shared purpose of making sense of scientific ideas and practices (e.g.,
everyone will reason and respond; challenge ideas, not identities). Discourse moves (e.g.,
inviting students to build on a classmate’s idea or summarize a key idea from a discussion) can
help shift responsibility for thinking to students. Decision rules provide teachers guidance on
how to proceed on the basis of assessment information. For example, if the class is divided
between two alternative explanations, a teacher might break the class into two groups and ask
students from each to pose questions to the other group about their explanations. Preliminary
indications suggest promising uptake by teachers of these dialogic supports:
“I really like the norms and I think they set a good standard in the classroom
of what do to, and I see the kids using it in everything they're doing... The
one I like the most is explaining, it's really making the kids support their
reasoning.” [Teacher teleconference, 2010/10/27]
“I agree, it has really set a good tone for the kids... There are a lot of times
when something is happening in class and I can refer back to those norms.
Like the one norm about "it's okay to be wrong" based on your current
understanding; that has come up on several occasions... We look at when
you took that test that was what your understanding was... but now that you
have this new understanding it’s different.” [Teacher teleconference,
2010/10/27]
4
Schank, SRI International
NSF Grant IIS-0713711
Bridging the Gaps to Better Support Agile Teaching in Real Classrooms
Among lessons learned, our research highlights the need to complement a focus on patterns
with attention to the style with which teachers enact them. We have progressed toward more
atomic, actionable moves as a way to help teachers to bridge the gap between theory and practice
so that they can innovate more productively in the classroom. These findings align with research
by colleagues in Spain who are also trying to facilitate highly interactive discussion in the
classroom using Group Scribbles. They observed that teacher lesson designs, despite being highlevel and often implicit in nature (i.e. not exhaustively specified on paper), followed a limited set
of atomic design patterns1 when analyzed (Prieto et al., 2010). Teachers’ enactments of those
designs also followed a further set of atomic patterns. The patterns shared a common trait in that
they were more easily translatable to concrete actions with the Group Scribbles tool and in the
classroom. Research efforts that tried to modify teachers’ practices by exposing them to
researcher-made abstractions (e.g. in training sessions, or through web 2.0 platforms) without
direct relationship and immediate benefits to their everyday practice generally did not succeed.
However, the use of the uncovered atomic patterns elicited from teachers’ practice proved much
more successful. For example, in a professional development workshop in which patterns elicited
from teacher practice were presented, the teachers were better able to use these patterns as a
starting point, design new activities with Group Scribbles that used these patterns, and reflect on
how the activities could be enacted in their classrooms and which ones may be more useful from
a pedagogical standpoint.
The pattern approach to scaffold teacher enactment and enable agile teaching with classroom
network technologies like Group Scribbles offers several advantages: it serves as a means of
communication between researchers and practitioners (and also among practitioners), it offers
practitioners a number of building blocks that can be creatively combined into new solutions,
and it is suitable for non-experts because of its problem orientation. However, the evidence from
our research (and from the research of our Spanish colleagues) shows that its application to
everyday teaching practice is not without potential limitations. Here we highlight two important
tensions or gaps that often arise when researchers try to influence everyday practice in an
authentic setting.
The first of these tensions appears between researchers' efforts to de-contextualize empirical
data to obtain widely applicable principles and teachers' application of those principles to
concrete situations, which can be seen as an act of re-contextualization (Goodwin & Duranti,
1992). Teaching practice, and especially innovative teaching practice, can be seen as the
appropriation of the de-contextualized tools such as curriculum materials, classroom
management techniques, as well as resources provided by researchers (e.g. theories, pedagogical
patterns, or even technological tools such as Group Scribbles). In this sense, pedagogical patterns
provide de-contextualized advice on how to attain certain pedagogical goals. However, as noted
by Winters & Mor (2009), dealing with de-contextualized tools can be difficult for teachers, even
if they contain more elements of context than an abstract theory (e.g. they assume a certain kind
of classroom, or they are provided along with a short narrative example of their application to
other contexts). Instructional moves and design and enactment routines elicited from actual
teaching practice are also examples of this de-contextualization effort, but they originate from a
1
The researchers called the activity patterns extracted from actual teaching routines (both design routines and
enactment routines) to differentiate them from researcher-specified patterns.
5
Schank, SRI International
NSF Grant IIS-0713711
different source than theory. Having more elements of a familiar context present in these patterns
(e.g. assuming usage of the Group Scribbles tool and a whiteboard, or assuming a specific
outcome of a previous task) enhances their mirroring properties and makes them more actionable
(i.e. teachers recognize them as actions that they normally take in the classroom or as actions
they may take in an easily recognizable situation).
There is also a second tension or gap between the macro-level designs and plans for
instruction (provided by researchers, developed by teachers themselves or co-designed) and the
emergent micro-level enactment of those plans by a specific teacher in his classroom. Such plans
are incomplete by their very nature, since any representation of a practice is a simplification.
Even if plans are designed by teachers thinking about their specific classroom context, plans
cannot take into account all emergent occurrences or accurately predict students’ notions and
their evolution. If we look at this gap from the point of view of socio-cultural activity theory
(Engeström, 1987), lesson plans and pedagogical patterns provide mediational tools for teachers
at the action level that respond to needs such as providing feedback to students or promoting
self-regulation. However, even with that scaffolding, teachers have still to make decisions on
how to exactly enact the plans using specific instructional moves in their classroom context.
These instructional moves (which correspond to operations in activity theory terminology) can
be highly routinized and often vary based on teacher style. By also providing scaffolding at the
operation level (e.g. discourse moves, classroom norms, decision rules), we propose that more
coherent pedagogical strategies can be enacted by teachers.
Having a set of atomic, actionable patterns that are easy to call forth, tweak and recombine
can empower teachers to creatively design and enact activities according to the theories and
design principles of CSCL research (Hernández et al., 2010) and dialogic teaching research
(Wells & Mejia-Arauz, 2006; O'Connor & Michaels, 2007). Moreover, this combination of
patterns of different granularities is supported by Alexander’s concept of a pattern language
(Alexander et al., 1977), that is, a set of related patterns that provide increasing detail on how to
implement the higher-granularity patterns. In our case, norms, rules, moves, and routines can be
seen as tools for goal-directed action that ideally become operationalized in ways that support
their enactment of collaborative, dialogic activities. The use of practice-derived enactment
patterns, classroom norms, and decision rules seems to complement the advantages of macrolevel pedagogical patterns and to enable a wider adoption and change of daily practice. We posit
the combination of both kinds of patterns as a coherent mediational strategy for teachers to
produce contextual, pedagogically-sound uses of technology, making the most of its affordances
for supporting deeper collaborative knowledge building.
6
Schank, SRI International
NSF Grant IIS-0713711
References
Alexander, C., Ishikawa, S., & Silverstein, M. (1977). A pattern language: Towns, buildings,
construction. New York, NY: Oxford University Press.
DeBarger, A., Penuel, W., Harris, C., & Schank, P. (2010). Teaching routines to enhance
collaboration using classroom network technology. In F. Pozzi & D. Persico (Eds.),
Techniques for fostering collaboration in online learning communities: Theoretical and
practical perspectives (pp. 224-244). Hershey, PA: IGI Global.
Engeström, Y. (1987). Learning by expanding: an activity-theoretical approach to
developmental research. Helsinki, Finland: Orienta-Konsultit.
Goodwin (Eds.), Rethinking context: Language as an interactive phenomenon (pp. 1-42). New
York: Cambridge University Press.
Hernández, D., Asensio, J. I., Dimitriadis, Y., & Villasclaras, E..D. (2010). Pattern languages
for generating CSCL scripts: From a conceptual model to the design of a real situation. In
P. Goodyear & S. Retalis (Eds.), E-learning, design patterns and pattern languages (pp.
46-64). Rotterdam, The Netherlands: Sense Publishers.
O'Connor, C., & Michaels, S. (2007). When is dialogue “dialogic”? Human Development, 50,
275-285.
Prieto, L. P., Villagrá-Sobrino, S., Dimitriadis, Y., Jorrín-Abellán, I., Martínez-Monés, A.,
Anguita-Martínez, R. (2010). Recurrent routines in the classroom madness: pushing
patterns past the design phase. Proceedings of the 7th International Conference on
Networked Learning (pp. 499-507). Lancaster, UK: Lancaster University.
Wells, G., & Mejia-Arauz, R. (2006). Dialogue in the classroom. Journal of the Learning
Sciences, 15(3), 379-428.
Winters, N., & Mor, Y. (2009). Dealing with abstraction: Case study generalisation as a method
for eliciting design patterns. Computers in Human Behavior, 25(5), 1079-1088.
7