Towards a Performative Body Mapping Approach
Petra Gemeinboeck1 and Rob Saunders2
Abstract. This paper presents a proposal for a creative robotics
approach to human-robot interaction. The ‘Performative Body
Mapping’ method exploits the expertise of artists and performers
to imagining novel robot morphologies and movements. The
proposed approach describes a mapping method between human
and robot bodies, which supports the learning of socially
meaningful interactions through imitation.
1 INTRODUCTION
This paper proposes a Creative Robotics approach to humanrobot interaction that explores non-humanlike morphologies for
robots and their capacity to move and behave in ways that are
socially meaningful to humans.
Creative Robotics is an emerging research area in
experimental arts that brings together methods from a range of
fields including robotic art, sculpture, interactive art, robotics
and artificial intelligence. Embedded within a critical discourse,
Creative Robotics looks at human-robot interaction from a broad
cultural perspective and develops new artistic practices for
producing and probing meaningful relationships. As such,
Creative Robotics is in direct dialogue with the field of HumanRobot Interaction.
Artists have innovated in robotics to create ‘living’ sculptures
and machine environments since decades. Important examples
include Ihnatowicz’s pioneering work The Senster (1970) that
moved in response to movements and sounds in its environment.
Using an approach that came to dominate robotics research from
the late 1980s onwards, The Senster implemented a small set of
simple behaviours that combined to produce seemingly more
complex ones. With Petit Mal, Simon Penny aimed to produce a
robotic artwork, “which was neither anthropomorphic nor
zoomorphic, but which was unique to its physical and electronic
nature” [7]. Mari Velonaki’s Fish-Bird, comprising two robot
wheelchairs developed in association with the Australian Centre
for Field Robotics, demonstrated that irrespective of the robots’
form their behaviours could convey a social capacity [14]. Both
authors have collaborated on Accomplice, a large-scale robotic
installation that turns the walls of a gallery into a playground for
a colony of artificially curious machines (see Figure 1) [6]. One
could argue that over the past 50 years, robotic art has
demonstrated that movement, more so than appearance, is key to
human recognition of a robot’s responsive and social qualities.
________________________________
1
Creative Robotics Lab, National Institute of Experimental Arts, College of
Fine Arts, University of New South Wales, Sydney, Australia. Email:
[email protected]
2
Design Lab, Faculty of Architecture, Design and Planning, University of
Sydney, Sydney, Australia. Email: [email protected]…….…
…
The motivation for developing this Creative Robotics study is
to empirically test the hypothesis that movement is fundamental
to a robot's capacity to carry social agency. Currently, much
research in human–robot interaction is based on the assumption
that robots that appear human- or pet-like are easier for people to
relate to and communicate with [5]. In contrast, we seek to
imagine robot morphologies and movement abilities that don’t
mirror humanlike or animal-like bodies and behaviours. The
challenge then is to understand how this ‘strange’ robot body can
move and express itself in ways that humans can relate to and
feel comfortable with. Thus, the objective of this study is to
explore how to efficiently teach a non-humanlike robot to move
according to its own machine embodiment, whilst imbuing it
with a sensitivity for the shapes, rhythms and textures of human
movements and gestures.
The following section will introduce a human-robot
interaction methodology, currently being developed by the
authors, that focuses on a non-humanlike robot’s capacities of
movement, in order to open up a much wider range of potential
social robot morphologies. The authors would like to share these
ideas to gain feedback from the Human-Robot Interaction
community in the early stages of the research development.
Figure 1. Accomplice, detail.
2 PERFORMATIVE BODY MAPPING
The ‘Performative Body Mapping’ (PBM) methodology
addresses two core open questions that have emerged from
research in Human-Robot Interaction (HRI): how should a social
robot behave, and, directly related, how does a social robot look
like? In addressing these open questions, this research will
attempt to tackle two fundamental assumptions in HRI, as
identified by Dautenhahn [5]: the robot’s ability to interact
‘naturally’ and the need for a social robot to appear humanlike.
A robot’s embodiment and with it, its capacity to perceive
and act, significantly differs from the human, independently of
how humanlike it may appear. This capacity determines the
perceptual world in which the robot acts—its umwelt [13], thus
human and robot are each embedded in their own distinct umwelt
[17]. The PBM methodology proposed here aims to open up the
possibility for both human and robot to bodily negotiate their
two umwelts. Complementing expert knowledge and methods
from human-robot interaction with expert knowledge and
methods from interactive art, performance, choreography and
dramaturgy, the research will foreground meaning making
through movement and aesthetics of experience in human-robot
interaction.
The following describes our proposed PBM approach to the
design and development of sociable, non-humanlike robots,
comprising three phases and learning cycles.
Phase 1: Design, Build—Learning Cycle 1
The first phase focuses on the design of the robot morphology.
The underlying concept draws upon the ‘body sculptures’ of
renowned artist Rebecca Horn, in particular the ‘Mechanical
Body Fan’ (1974), a soft radial fan, stretched by metal ribs,
measuring three meters in diameter. The ‘Mechanical Body Fan’
was attached to a performer, whose movements reconfigured the
body fan, creating constellations of planes that seemingly mark
intersections between her body and space. The notion of ‘body
sculptures’ allows for the imagining of a body that is different to
the human in shape, structure, and ability to move, but whose
movements can still be negotiated by a human body (see
diagram of ‘Mechanical Body Fan’ illustrating a model for
possible non-humanlike robot morphologies in Figure 2). This
dramaturgical strategy was pioneered by Bauhaus artist and
theatre designer Oskar Schlemmer, who used geometric
costumes of substantial volume to transform dancers’ bodies and
constrain their movements (Figure 3). The costumes for the
‘Bauhaustänze’ opened-up “a range of possibilities to change
bodily relationships to the exterior space” and required the
dancers to “develop particular haptic sensibilities” [3].
lightweight
fabric
membrane
The design process begins with conceiving a non-humanlike
‘body sculpture’ that fits around, along or in relationship with a
human body. This is both a ‘sculptural costume’ for the
performer to inhabit and activate, and a prototype of the robot’s
eventual body. Criteria for the non-human morphology imagined
at this stage are: no obvious front and back; no head or face; no
limb-like structures. The objective is to experiment with a
geometric and relatively simple/abstract form or frame
consisting of articulate planes that effectively constrain and
redirect usual human modes of movement. Inhabiting or
performing the ‘body sculpture’ will allow a performer to enact
the robot’s body and to get a bodily sense for what it can do.
The contours and movement capacity of both this high
fidelity prototype (‘performed’ by human) and the robot’s standalone body will be virtually the same. For the robot body to
become an autonomous agent, the sculpture will be extended
with a sensorimotor system that will enable the robot to move its
articulated planes in a highly controlled fashion. The sensorimotor system will also include distance sensors to avoid
collisions with people and the environment, and wide-angle
cameras for detecting faces. These cameras will be hidden to
ensure that they will not be interpreted as eyes.
Figure 3. Triadic Ballet by Oskar Schlemmer
flexible rods
performer
Figure 2. Diagram of Mechanical Body Fan as a model for
possible non-humanlike robot morphologies
Following the prototyping and development of the robot
morphology, in the first learning cycle the robot will learn how
to develop a map of its body and how it can move in relation to
the environment through self-exploration. This initial selfexploratory learning will ground the robot’s later imitation
learning in its specific embodiment. At this stage, we imagine to
use ‘active motor babbling’ [11], a technique from
developmental robotics. The robot will learn a sensorimotor
mapping between its motor movements, proprioceptive sensors
and an external view provided by three 3-D depth cameras
(overhead and both sides) that produce a high-density pointcloud representation of the robot and its environment in realtime. This combined feed of external visual information will
permit the robot to learn an inverse mapping from body shape to
movement, providing a foundation for imitation learning [2].
Concluding this phase will be a workshop study with a
performer inhabiting and performing the high-fidelity
prototype/model of the robot body design. Here we will explore
and record a set of expressive movements and gestures that reenact everyday social encounters and interactions, as constrained
by the robot’s body contours. The workshop setting will use the
identical 3-D depth camera configuration used in the robot’s
self-exploratory learning to record the performer’s movements.
Phase 2: Imitation Learning, Curious Exploration—Cycle 2
In the second learning cycle, the robot will learn to imitate the
recorded movements from the performer, using the high-density
point-cloud representation recorded in the first workshop as its
video input, but with the performer’s human body digitally
erased. A significant aspect of the PBM method is that the input
for the robot’s learning is video of the performer inhabiting the
model of the robot body, at which times the robot will learn by
copying a moving body that looks like its own body. It is thus
expected that footage of the performer can be used to drive and
expedite the imitation learning. The sculpture approach, by
providing training data for the robot, becomes a liminal resource
for ‘bootstrapping’ the robot’s learning.
Once the robot has learned to imitate the given set of
movements and gestures from the workshop study in Cycle 1, it
will learn to extend and modify them based on what its different,
robotic embodiment affords it to do. Methods from
computational creativity—in particular the computational
modelling of curiosity, in which the motivation for an agent’s
open-ended learning is the learning itself rather than an
externally defined goal [12]—will provide the robot with the
motivation to explore combinations of and variations on its
movements. This self-motivated adaption of learned movements
will allow the robot to expand its repertoire of behaviours.
As part of this learning process, the robot will rehearse its
movements in a series of workshops involving the performer. In
a feedback loop indicative of social learning in wider society,
this process will be repeated as required. We expect the
performer to also learn from the robot in these workshops, and
each time the robot will again learn to imitate the recorded
movements by performer. The performer will also be asked at
each stage of the development process to provide feedback on
the morphology’s potential for expression and its ability to
successfully imitate the expressive qualities of the recorded
movements. If specific problems are identified, this may result in
alterations to the robot’s morphology, which in turn would be
reflected in the model performed by the performer.
Phase 3: Turn-taking, Social Interaction—Learning Cycle 3
The final phase introduces social interactions. The robot will
learn to move in direct response to a person’s movements and to
elicit response from a person in real-time, based on robot–human
interaction kinesics [9]. In a further series of workshops, the
performer will again inhabit the robot model (original
‘sculpture’), this time enacting the robot in improvising social
scenarios with another person (not in ‘costume’).
To facilitate the learning for the robot, these social scenarios
will be limited to turn-taking interactions [9], again captured by
the three 3-D depth cameras as a high-density data-cloud, and
again with the performer’s human body digitally erased. This 3D external view will provide the robot with the same
‘pedagogical’ set-up as in the previous cycles, with the addition
of an interlocutor with whom it needs to learn to correlate its
mappings. The next workshop will then involve the robot in
direct social interaction with a non-costumed human, attempting
its first socially interactive, turn-taking gestural conversations.
Turn-taking will permit the robot to put into practice its
repertoire of behaviours in engaging an interlocutor socially. The
robot will learn simple cue and response pattern and attending to
different rhythms and timings in its corporeal interaction, which
humans are sensitively attuned to in human-human interaction.
Another new skill that the robot will need to acquire in this
learning cycle is the ability to recognise simple gestures using a
Markov model [1] and respond appropriately based on its
learned repertoire of turn-taking interactions.
To further facilitate a more situated interaction between
human and robot, this phase will also include an expansion of
the robot’s reflexes, based on which it will be able to track
movements and attend to sounds in the environment [8]. While
these are simple automated responses, they provide the necessary
social scaffolding for the interaction.
Cycle 3 will conclude with a workshop with a group of
performers that serves as rehearsal for an open experiment study,
where the robot and its expressive movements and interaction
capabilities will be tested in a museum or gallery setting. During
the rehearsal workshop, a group of performers will engage with
the robot-sculpture to test how it can cope. This will provide
useful data to better understand the constraints needed for the
open experiment study.
Evaluation in Public Space
The efficacy of the methodology will be evaluated through two
studies based on the open experimentation method in creative
robotics [14] and the authors’ experience in tracking the
movements of audience members in interactive art exhibitions.
The objective for the public study is to involve non-expert
participants in an interaction scenario with the robot, without
them being prepared or instructed. The museum or gallery
setting is not only chosen for gaining access to a large number of
unprepared participants, but also for providing a ‘natural’ social
framing for the study. Studies on social interaction in museum
spaces have demonstrated that “social interaction forms a pivotal
and a virtually unavoidable part of people’s experience of
museums” [15].
In the museum, the robot will be presented as a kinetic
sculpture, not a social robot, to avoid audience members
encountering the robot with already–formed expectations.
Visitors, who approach the ‘sculpture’, will be surprised to be
attended to and greeted. One goal of this study is to observe
whether the robot will be able to sustain interaction with
individual members of the audience and to proactively elicit
responses from the audience. Is the ‘sculpture’ experienced as a
social agent? A more typical interaction scenario in the museum
context would be for the audience member to engage with the
robot to find out ‘how it works’ or to probe how it can be
controlled [4].
Quantitative video analysis will be deployed along multiple
dimensions, e.g. proximity and spatial alignment of audience
members, duration of engagement [16]. In addition, qualitative
results will be gathered using a questionnaire with a series of
open-ended questions related to the visitor’s social experience.
3 PROTOTYPING SOCIAL ROBOTS
The ‘Performative Body Mapping’ methodology builds on and
extends the Theatrical Robot methodology [10]. In the Theatrical
Robot method, an actor or mime artist is disguised as a robot,
“behaving according to a specific and pre-scripted robotic
behaviour repertoire” [5]. It is used to embody a live-sized robot
to simulate humanlike movements and cognition. The following
will describe how the methodology, presented here, expands on
the Theatrical Robot.
The PBM method involves performers in the early
prototyping of both the morphology and the robot’s behavioural
envelope through the use of low and high fidelity prototype
‘body sculptures’ worn by the performers. The performers
provide expert feedback on the expressive potential of the robot
morphology, which is not restricted to humanoid forms but
instead can take on any form that can be inhabited by or attached
to the performer, e.g., using of props/prosthetics. The performer
is not restricted to pre-scripted behaviours but instead is required
to improvise movements while taking on the character of a nonhumanlike robot. While the ‘body sculpture’ may impose
constraints on the performer, much of the value in the study
comes from the embodied knowledge that the performers
provide through improvisation.
The interactions in the performative body mapping may also
be staged, i.e., with another performer, such that a range of
responses to social situations may be recorded in controlled
conditions. Similar to the Theatrical Robot, this provides a way
of thinking through movement scenarios in an embodied way by
allowing experts within the domain, i.e., performers, to
communicate their embodied knowledge. Finally, by providing
training data for the robot, the ‘body sculpture’ is not a stand-in
for a robot that has not yet been built, but rather a resource for
‘bootstrapping’ the learning of a non-humanlike robot with
socially meaningful movements and interactions.
It is our hope that the morphological mapping between human
and robot bodies established within the PBM simplifies the socalled correspondence problem [2] as the robot learns to imitate
a human disguised and performing as that particular robot; thus
the human teacher ‘meets the robot half way’.
REFERENCES
[1] F. Bevilacqua and R. Muller. A Gesture Follower For Performing
Arts. In: S. Gibet, N. Courty and J-F. Kamp (eds.) Proceedings of
the 6th International Gesture Workshop (2005).
[2] A. Billard, S. Calinon, R. Dillmann, S. Schaal. Robot programming
by demonstration. In: Springer Handbook of Robotics, Springer
Berlin Heidelberg, pp. 1371-1394 (2008).
[3] J. Birringer. Bauhaus, Constructivism, Performance. In: PAJ. A
Journal of Performance and Art 35:2, 39-52 (2013).
[4] O. Bown, P. Gemeinboeck, R. Saunders. The Machine as
Autonomous Performer. In: L. Candy, S. Ferguson (eds.) Interactive
Experience in the Digital Age. Evaluating New Art Practice.
Springer, 75-90 (2014).
[5] K. Dautenhahn. Human-Robot Interaction. In: Soegaard, Mads and
Dam, Rikke Friis (eds.) The Encyclopedia of Human-Computer
Interaction, 2nd Ed. Aarhus, Denmark: The Interaction Design
Foundation, http://www.interaction-design.org/encyclopedia/humanrobot_interaction.html (2013).
[6] P. Gemeinboeck and R. Saunders. Creative Machine Performance:
Computational Creativity and Robotic Art. In: Maher, M.L., Veale,
T., Saunders, R. and Bown, O. (eds.) Proceedings of the Fourth
International Conference on Computational Creativity, University
of Sydney, Australia, 215–219 (2013).
[7] S. Penny. Embodied Cultural Agents: At the Intersection of Art,
Robotics, and Cognitive Science. In: Socially Intelligent Agents:
Papers from the AAAI Fall Symposium, AAAI Press (1997).
[8] R. Pfeifer and J. Bongard. How the body shapes the way we think —
a new view of intelligence. MIT Press (2007).
[9] B. Robins, K. Dautenhahn, C.L. Nehaniv, N.A. Mirza, D. Francois
and L. Olsson. Sustaining interaction dynamics and engagement in
dyadic child-robot interaction kinesics: Lessons learnt from an
exploratory study. In: Procs of the 14th IEEE Int. Workshop on
Robot & Human Interactice Communication (RO-MAN 2005), 716722 (2005).
[10] B. Robins, K. Dautenhahn and J. Dubowski. Investigating Autistic
children's attitudes towards strangers with the theatrical robot-A new
experimental paradigm in human-robot interaction studies. In: Procs
of the 13th IEEE International Workshop on Robot and Human
Interactive Communication (RO-MAN 2004), 557-562 (2004).
[11] R. Saegusa, G. Metta, G. Sandini and S. Sakka. Active motor
babbling for sensorimotor learning. In: Procs of the IEEE
International Conference on Robotics and Biomimetics (ROBIO
2008), 794-799 (2008).
[12] R. Saunders. Towards Autonomous Creative Systems: A
Computational Approach. In: Cognitive Computation, Special Issue
on Computational Creativity, Intelligence and Autonomy, 4:3, 216–
225 (2012).
[13] J.v. Uexküll. A Stroll Through the Worlds of Animals and Men: A
Picture Book of Invisible Worlds. In: Claire H. Schiller (ed. and
trans.) Instinctive Behavior: The Development of a Modern Concept.
New York: International Universities Press, Inc., 5–80 (1957).
[14] M. Velonaki. Multi-Objective Evaluation Of Cross-Disciplinary
Experimental Research. In: Studies in Material Thinking 8, AUT
University, http://www.materialthinking.org (2012).
[15] D. vom Lehn, C. Heath and J. Hindmarsh. Re-thinking Interactivity.
In: Rethinking Technologies in Museums. Limerick, Ireland, 29 June
2005.
[16] M. L. Walters, K. Dautenhahn, S. N. Woods, K. L. Koay, R. Te
Boekhorst, and D. Lee. Exploratory studies on social spaces between
humans and a mechanical-looking robot. In: Connection
Science, 18:4, 429-439 (2006).
[17] T. Ziemke, N. Sharkey. A stroll through the worlds of robots and
animals: Applying Jakob von Uexküll's theory of meaning to
adaptive robots and artificial life. In: Semiotica, 134:1-4, 701-746
(2001).