Wriggle!
Creating a Platform for Dynamic and Expressive SocialEmotional Play
Katherine Isbister1, Rainey Straus2, Jennifer Ash3
1
Games Research Laboratory
Rensselaer Polytechnic Institute
Winslow Building 3rd Floor
105 8th Street
Troy, NY 12180 USA
[email protected]
2
406 49th Street
Oakland, CA 94609 USA
[email protected]
ABSTRACT
This paper describes the research rationale and early prototyping environment being developed for Wriggle!— a physically
and emotionally engaging game experience and accompanying
exhibition and research project. Wriggle! is meant to enable
exploration of the social and dynamic nature of emotions, as
well as leveraging the appeal of games to enhance comfort
with exploring engineering skills and methods.
Author Keywords
Emotion, affect, gesture, social, digital game, do-it-yourself,
maker culture
INTRODUCTION
Wriggle! is a project that combines artistic practice,
game design, and HCI theory and research, as well as
museum exhibition and curatorial efforts. The core concept is a game environment that allows multiple players
to use their body movement to create expressive and
interactive traces. The play parameters will be designed
to encourage players to explore how movement can express and create emotional atmosphere, and can become
an alternate form of emotional communication. These
images of Pollock and Picasso using gesture to create
traces (figures 1 and 2) evoke the kind of dynamic action
we hope to create with the game space—something we
felt fit the Supple Interface workshop call well [11].
3
Games Research Laboratory
Rensselaer Polytechnic Institute
Winslow Building 3rd Floor
105 8th Street
Troy, NY 12180 USA
[email protected]
hance the experience and underlying approaches to emotion and communication that are embodied in the game,
as well as the do-it-yourself mode of practice that we
hope to encourage in visitors themselves after they leave
the exhibit hall.
Theoretical Framing
Wriggle! rests upon a growing body of theory within and
related to the HCI field, which underscores the importance of the body in interaction, particularly in circumstances in which emotion comes into play.
Increasingly, those who study emotion as a mental phenomenon have made the argument that how we make
sense of the world and our feelings is strongly tied to our
physical selves (e.g. [6]). HCI theorists such as McCarthy and Wright [18, 19] and Dourish [7] have brought
notions of the body and embodiment to the forefront of
considerations in understanding digitally mediated experiences, as have new media theorists such as Hansen [8].
Designing with the body in mind is becoming a growing
part of practicing designers’ discussions (e.g. [15]).
Meanwhile a growing number of system builders have
worked to imbue their creations with considerations of
the embodied nature of emotion (e.g. [2, 21, 12].), exploring ways in which emotion is engaged through engaging the body of the ‘user’ of a system.
Wriggle! builds upon this work around body and emotion, with a particular aim—allowing players to viscerally experience the social and dynamic nature of emotion
through the gaming experience. Wright and McCarthy
point to the social nature of sense-making around experience [18], and social psychologists point out that emotion is a highly ‘contagious’ and socially constructed
phenomenon [9, 16]. With Wriggle! we hope to create
engaging gestural play among participants that helps
raise their meta-awareness that we all exist in a codetermined emotional atmosphere, and to begin to examine the ways in which they and their fellow players influence that atmosphere.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Figures 1 and 2. Evocative traces created by artists
Pollock and Picasso.
The Wriggle! project has been conceived from the beginning as something that can be hosted in venues like
the Esther M. Klein Gallery in Philadelphia [14], which
present works on the boundary between the arts and sciences. We have been working with the Klein Gallery to
plan curatorial copy and related events that help to en1
which show some of the types of shape factors that can
come into play.
Figure 3. A slide from Phil Harrison’s Game Developers Conference Keynote about ‘Game 3.0’.
The Wriggle! project also draws upon a growing emphasis within the HCI, game development, and larger design
community toward empowering everyday consumers to
craft their own experiences. Make magazine is an example of a rallying point for amateur creators, which in turn
points to live events as well such as the “Maker Faire”
[17]. Sony’s Phil Harrison gave a keynote at this year’s
Game Developer Conference in which he touted ‘Game
3.0’, with an emphasis on ‘online collaboration and user
generated content’ [20]. Research labs such as Scott
Klemmer’s at Stanford (http://hci.stanford.edu/srk/) take
a bottom-up approach to creating tools and toolkits for
designers that use this maker aesthetic. Wriggle! takes
this approach into the games arena, making use of the
Nintendo Wii controller (Figure 4), but driven using a
Max/MSP/Jitter patch.
Figure 5. The Sensual Evaluation Instrument objects,
created by Straus for a prior research project, illustrate the sorts of form factors we might leverage
when creating the ‘skins’ for the controller.
By taking a mass-market console-based game remote
and repurposing it for a very different environment and
providing it with a very different ‘skin’, we are hoping to
show gallery goers and the larger player community that
they, too, can ‘hack’ the consumer play environment to
produce radical new experiences for themselves.
This do-it-yourself message will also be a part of the
Klein Gallery’s larger curatorial plan for the exhibition
of Wriggle!
Project context:
The concept for Wriggle! emerged from three key
streams of work among the authors:
1. An ongoing arts practice in which Isbister and Straus
have used player content creation strategies in the Sims
games to produce artworks (see www.simgallery.net for
an overview of this work).
2. Exploration of affective systems and their evaluation,
in collaboration with colleagues at the Swedish Institute
for Computer Science [12].
3. Exploration of nonverbal communication in digital
interactions [10, 13].
Goals of the project:
It is our hope that Wriggle! will provide:
- A powerful and engaging play experience for those
who try it out. We are inspired by experiential goals [3
(4), 22] set out by the vanguard of the independent gaming community.
Figure 4. The Nintendo Wii controller.
Max/MSP/Jitter is a long-standing graphical programming environment based upon MIDI, with a large and
very active development community (see [5]). Within
months of the release of the Nintendo Wii system, Max
users had developed several patches that allowed users
to access the Wii controller. We are using one called
aka.wiiremote, developed by Masayuki Akamatsu.
We are hoping to develop ‘skins’ for the controller that
shift its form factor and create interesting emotional affordances for the player. Straus has sculpted biomorphic
objects for a prior project [12] shown here in Figure 5,
2
- An opportunity for us to meld and apply our artistic
and design practice to create a compelling platform for
play.
- An inspiration to others who want to repurpose consumer hardware.
- Through play data and player-authored mini-games, a
new window into gestural dynamics and emotional
communication
DESIGN OVERVIEW
Wriggle! consists of several interwoven components:
1. An on-site gaming experience. Players will manipulate sensor-enabled objects (built up from the Wii controller and possibly also the Nike shoe sensor, with
sculptural elements layered onto them) to explore creat-
ing shared emotional atmosphere in several mini-games
(see Figure 6). The game space will be displayed on a
large projection screen and audio will be played through
speakers in the installation (and potentially also through
the game controllers’ small speakers as well).
Figure 6. An early sketch of the components of the game experience.
2. A web-based auxiliary gaming experience and collaboration space. We hope to build a way for remote participants to join in with mouse-based interaction. In either
case we will provide a web environment in which visitors can learn about the exhibit, learn about do-ityourself component hacking and concepts, and have an
opportunity to add mini-games to the Wriggle! repertoire
that they have built themselves.
3. A database of emotional dynamics collected from
gaming during the exhibition. The project is designed to
have minigames that are in essence ‘calibration’ exercises in asking players to act out emotions and to explore
social psychological effects such as emotional contagion. Their play sessions will be recorded so that we can
build a deeper parameterized knowledge of social emotional dynamics toward applications in other areas of
experience design, and towards evolution of our basic
understanding of shared emotional dynamics.
4. On-site curatorial text and events, including a do-ityourself workshop for teaching young people how to
‘hack’ comparable systems and how to add content to
Wriggle! The Klein Gallery will craft text and a series
3
of events that help to frame the notion of emotional expression and its translation into a new vocabulary in the
digital realm, and the culture of hacking these new social
and emotional territories. Isbister will also work with
engineering students and faculty at RPI to coordinate
DIY workshops for locals to learn about how to repurpose consumer components such as the Wii controller and Nike shoe sensor, toward empowering them to
considering engineering as a way to ‘hack the social
atmosphere’.
CURRENT STATE OF WRIGGLE!
The Wriggle! team has done early brainstorming of gestural play possibilities, using Isbister’s Game Research
Lab motion capture system (see Figure 7). This semester, Ash (an undergraduate at RPI) has been taking an
interactive arts programming course, and put together
the working demonstration of the Wii controller patched
to a drawing program, that we’ll be showing in the
workshop. We are planning to collect initial feedback
from workshop attendees about what they like and would
like to see in terms of gestural input and reactivity from
the system, with a follow-up game design workshop
planned for May. We hope to involve a broad community of game designers in the project, which is possible
due to the easy accessibility of both the controller and
the Max/MSP/Jitter patches.
4. Csikszentmihalyi, M. Beyond Boredom and Anxiety: Experiencing Flow in Work and Play. San Francisco: JosseyBass, 1975.
5. http://www.cycling74.com/
6. Damasio, A. The Feeling of What Happens: Body and
Emotion in the Making of Consciousness. San Diego: Harcourt, Inc., 1999.
7. Dourish, P. Where the Action Is: The Foundations of Embodied Interaction. MIT Press, 2001.
8. Hansen, M.B.N. Affect as interface: Confronting the ‘digital facial image’, Chapter 4 in New Philosophy for New
Media, MIT Press, 2004.
9. Hatfield, E., Cacioppo, J.T., Rapson, R.L. Emotional Contagion. Cambridge University Press, 1994.
10. Isbister, K. Better Game Characters by Design: A Psychological Approach. Morgan Kaufmann, 2006.
11. Isbister, K. and Höök, K. Supple Interfaces: Designing and
evaluating for richer human connections and experiences.
Proc. ACM SIGCHI’07 Workshop. ACM Press. 2007.
12. Isbister, K., Höök, K., Sharp, M. and Laaksolahti, J. The
Sensual Evaluation Instrument: Developing an Affective
Evaluation Tool. Proc. ACM SIGCHI 2006, ACM Press.
2006.
13. Isbister, K., and Nass, C. Consistency of personality in
interactive characters: Verbal cues, non-verbal cues, and
user characteristics. International journal of humancomputer studies 53(2), 251-267, 2000.
Figure 7. Gestural play brainstorming using a motion
capture system. (Movie is available here:
http://wriggle.hss.rpi.edu/wriggle.hss.rpi.edu/images/
c/c4/WriggleEdit.mov )
CONCLUDING REMARKS
Wriggle! is an adventurous interdisciplinary experiment,
which we hope will provide both a compelling end experience, and some valuable insight into social emotional
dynamics. We invite your reactions to both the concepts
and the initial controller prototype, toward a stronger
final design that is truly ‘supple’.
ACKNOWLEDGMENTS
We would like to thank Curtis Bahn and Tomie Hahn of
RPI’s Arts Department for their insights and assistance;
Noah Schaffer, a Ph.D. student at RPI, and Rob Ecuyer,
the technical support person for the Games Research
Lab, for their assistance in running the motion capture
brainstorm and helping us acquire the very elusive Wii
system. Thanks also to Kirsten Boehner of Cornell for
joining in our brainstorm.
REFERENCES
1. Bahn,
C.
Interactive
Arts
http://www.arts.rpi.edu/crb/iap/index.html
14. Esther M. Klein Gallery, http://www.kleinartgallery.org/
15. Klemmer, S.R., Hartmann, B., and Takayama, L. How
bodies matter: Five themes for interaction design.
DIS 2006: ACM Conference on Designing Interactive
Systems: pp. 140–149.
16. Knapp, M.L. and Hall, J.A. Nonverbal Communication in
Human Interaction. Wadsworth Thomson Learning, 2002.
17. http://www.makezine.com
18. McCarthy, J.C. and Wright, P.C. Making sense of experience. Chapter 5 in Technology as Experience, MIT Press,
2004.
19. McCarthy, J.C. and Wright, P.C. The enchantments
of technology, pp. 81-90 in Funology: From Usability to Enjoyment, M.A. blithe, K. Overbeeke, A.F.
Monk, and P.C. Wright editors, 2003.
20. Radd, D. Gaming 3.0: Sony's Phil Harrison explains
the PS3 virtual community, Home. BusinessWeek.com,
March
9,
2007.
http://www.businessweek.com/innovate/content/mar2
007/id20070309_764852.htm?chan=innovation_gam
e+room_top+stories
Programming,
21. Sundström, P., Ståhl, A., Höök, K., In Situ Inform-
2. Boehner, K., DePaula, R., Dourish, P., Sengers, P. How
emotion is made and measured. International Journal of
Human Computer Studies 65(4): 275-291.
ants Exploring an emotional Mobile Meassaging System in Their Everyday Practice, In a special issue of
IJHCS on Evaluating Affective Interfaces, vol. 65(4):
388--403, 2007.
3. Chen, J. Design for Entertainment, Innovation in Indie
Games Panel, Independent Games Summit, Game Developers Conference San Francisco, CA, 2007.
http://interactive.usc.edu/members/jchen/IGS%20Innovati
on%20Slides.swf
4
22. Swink,
S.
Principles
of
Virtual
Sensation,
http://www.steveswink.com/principles-of-virtualsensation/
5