Proceedings of the ASME 2013 International Design Engineering Technical Conferences and
Computers and Information in Engineering Conference
IDETC/CIE 2013
August 4-7, 2013, Portland, Oregon, USA
DETC2013-13319
ONLINE IMPLEMENTATION OF THE DELTA DESIGN GAME FOR ANALYZING
COLLABORATIVE TEAM PRACTICES
Sharad Oberoi
The Robotics Institute
Carnegie Mellon University
Pittsburgh, Pennsylvania 15213, U.S.A
Susan Finger
Department of Civil & Environmental Engineering
Carnegie Mellon University
Pittsburgh, Pennsylvania 15213, U.S.A
Eric Rosé
Institute for Software Research
Carnegie Mellon University
Pittsburgh, Pennsylvania 15213, U.S.A
ABSTRACT
Over the past four years the authors have developed an
online version of the Delta Design game, a board game which
was developed by Bucciarelli (1) to teach students design
collaboration skills. In the online version, players move tiles on
a shared virtual board and communicate only through text chat.
In addition, the objective functions are computed automatically
each time a tile is moved, so the focus of the game changes
from rapid number-crunching to negotiation. Since every state
of the board, along with micro-level team performance and chat
data, are captured, the resulting corpus from 38 four-player
team games provides a rich resource to explore different
aspects of collaborative team practices.
This paper gives an overview of the online implementation
of Delta Design and discusses the findings from user studies
including several undergraduate capstone design classes.
Observations of the board-moving tactics show that teams
planning a strategy before starting the game or players sharing
details about their role’s constraints with other team members
do not have much effect on the game’s outcome. Finally, this
paper demonstrates that the complex rules of the Delta Design
game make it a suitable candidate for analyzing collaboration
strategies in team-based design projects.
INTRODUCTION
In the 1980s, MIT Professor Louis Bucciarelli created the
Delta Design board game to engage undergraduate engineering
students in a team design project requiring communication and
negotiation (1, 2). The board game lasts for 1 to 1.5 hours and
involves four students in the roles of architect, project manager,
thermal engineer and structural engineer. The interdependence
of the objectives of the roles requires the players to reveal their
constraints in order to achieve a satisfactory outcome. Because
the computations required to evaluate the objective function for
each role are lengthy, and because they must be recomputed
each time a board tile is moved, even successful teams barely
manage to achieve most of their design objectives in the
allotted time. In addition, owing to the physical and temporal
nature of the game, key attributes of collaboration and the
negotiations within the team cannot be captured adequately.
This prevents researchers from analyzing collaborative team
practices during the game.
In order to capture the negotiation, the authors created an
online version of the game in which players move tiles on a
shared virtual Delta Design board and communicate only
through text chat. In the online game, the objective functions
are computed automatically each time a tile moves, so the focus
of the game changes from rapid number-crunching to
negotiation. During play, for every valid state of the board, each
player can see a detailed personal state as well as qualitative
feedback - red, yellow and green lights - on the objective
functions of each of the other team members. Using the online
game, differences between teams with effective negotiation
strategies and those with ineffective ones become easier to
analyze and understand.
Using text chat as the method of communication between
students (vis-à-vis video and voice chat) allows for easy
repeatability of the experiment over the years by reducing the
amount of data pre-processing required. Our experience with
1
Copyright © 2013 by ASME
automated voice transcription suggests that it introduces an
extra source of errors and requires additional resources for
human annotation and correction of the machine transcriptions.
On the other hand, without video/voice data we cannot capture
the subtle body language cues and changes in vocal tones that
might indicate tensions or a sense of urgency among
participants. However, since all communication goes through
text chat, students express their emotions (any sense of
approval/disapproval about the game moves) explicitly in
writing, making it easier for us to analyze the extent of their
collaboration.
BACKGROUND
The Delta Design game involves four players, each of
whom receives two sets of instructions. The first set describes
the common goal of the team and the design task with detailed
design requirements from the clients. The second set is specific
to each role and describes the functional requirements of the
role and how to meet them. Some of these requirements are
precise and easy to calculate, while others are imprecise and
subjective.
Design Objectives of Delta Design
In Delta Design, the life on planet DeltaP is different from
that on earth. DeltaP is a planned flat world with the design
done in a diagonal 2-dimensional space. The team designs the
residence for the residents of DeltaP using red and blue deltas
shaped like equilateral triangles. The red deltas provide heat,
while the blue deltas have a cooling effect. Due to their
different thermal and aesthetic properties, the design of the
residence is complex. Each team is asked to design a house that
meets all the structural, thermal and aesthetic constraints as
well as taking into account the physical limitations of the
construction materials. Moreover, since the DeltaP design
environment and its skewed coordinate system is equally
unfamiliar to all players, it homogenizes their prior technical
knowledge.
Delta Design Collaboration
A Delta Design team has a project manager, a structural
engineer, a thermal engineer and an architect. Since each player
only knows the rules for his/her own role, the game has an
inherent bias in favor of more communicative teams that can
better negotiate among themselves. The rules are also designed
so that certain roles can easily form alliances, while others have
conflicting objectives. For example, the thermal engineer does
not want to have more than two red (or blue) deltas connected
anywhere since they can overheat (or overcool). On the other
hand, the project manager wants as many blue deltas connected
together as possible since the cement used in blue-blue joints is
the least expensive. In addition, the architect wants to have blue
deltas dispersed throughout the house since 100 % blue
dispersion is one of his role’s design objectives. Thus, the
thermal engineer and architect would be more inclined to have
increased blue dispersion than the project manager. Unless the
team realizes that they need to see the big picture and
compromise, they cannot be successful in the game.
Delta Design Game as a Research Testbed
The Delta Design game has been used by universities all
over the world. Instructors have generally used the original
game to emphasize the role of negotiations in achieving shared
objectives. Researchers have adapted the game to teach
collaborative design practices to students, such as designing a
space shuttle built in Lego® blocks with functions and rules
assigned for several students (3), as a practical experience of
ethical decision-making in the engineering design process (4),
changing the player roles to emphasize principles of solid
mechanics (5), adding new representations and conceptions for
different players to simulate a real-world environment (6).
Prior research has also evaluated Delta Design from
different perspectives. Brandt discussed Delta Design as an
example of exploratory design games that are gaining
popularity to demonstrate collaboration between individuals in
participatory design projects (7). Besterfield-Sacre et al. used it
as a testbed for observing students’ performance through
behavioral observation (8). Svihla et al. used Delta Design to
demonstrate the increasing role of distributed expertise and
authenticity in the development of design expertise (9).
In this research, the authors have used the original game as
a microcosm of a real-world project. Since all the interactions
between students are captured through the online version of the
game, this allows a unique insight into how designers
collaborate.
APPROACH
Delta Design Application
The rules of the board-based Delta Design game, as listed
in the primers for all the roles (structural engineer, thermal
engineer, architect and project manager), have been adapted as
the functional requirements for the online game. In addition,
every action taken on the shared board and all the text chat
between participants are recorded. The players can roll back the
canvas to a prior configuration if they realize that they have
made a wrong move. This feature is useful since it allows the
players to consider various strategies to maximize their scores.
Both the automatic computation of the objective functions and
the ability to roll back the virtual board change the nature of the
collaboration from the physical game play.
The individual parts of the board are shown in Figure 1.
The toolbar includes tools required to play the Delta Design
game. The calculation buttons show detailed results of the
quantitative measures of design quality for the individual
players (not all qualitative measures, such as the cragginess of
internal design for the architect, were modeled). The canvas is
the area where the tiles are placed for playing Delta Design.
The contact list includes names of the team members, their
assigned roles and their online status (to check whether they
have connected to the game properly). The IM chat client
allows instant text messaging with other members of the team.
Only one person is allowed to work on the board at a time.
In order to move a tile the player needs to lock the board first.
The other players can see what is happening in real-time but
cannot move any tiles. The board can be locked for up to 3
2
Copyright © 2013 by ASME
minutes at a time. When a team-member has locked the board,
the other members see a red border around the board, along
with a prompt about who is in control. The players can send
comments and requests to their team members in the chat
window, but cannot talk to them in person. In order to win,
everyone on a team must meet his/her goal. Every player has
met his goal when all the stop lights on the tabs are green. A
player within 10% of the goal for any one of his conditions and
had met all the other conditions is assigned a yellow light. This
Toolbar icons
allows other members of the team to see how their moves on
the canvas affect the performance of the team as a whole.
Application Design
The playing field is composed of diamond shaped units
which are laid out on a coordinate system on a 150° angle
(instead of a standard 90° angle for a traditional Cartesian
coordinate system). This system can be seen as a grid
composed of triangles called “quads,” each of which bisect one
of the diamonds. Each delta consists of four smaller quads, as
Calculation buttons
Contact list of
team-members
IM chat client
Canvas
Figure 1. Screenshot of the online implementation of Delta Design
shown in Figure 2. The “origin” or center of a delta is
considered to be the middle quad of the four. With this
designation, each delta can be represented using three
components: an x-coordinate, a y-coordinate, and an
orientation. Figure 3 shows that the first delta (21,9, down) can
be expanded into the following list of quads: [(20,9), (21,9),
(22,9), (21,10)]. The second delta (24,9, up) can be expanded
into the following list: [(23,9), (24,9), (25,9), (24,8)].
The placement of deltas on this grid follows certain rules
which vary with the orientation. For “up” deltas, the x
coordinate must be odd if the y-coordinate is even. Conversely,
the x coordinate must be even if the y-coordinate is odd. For
“down” deltas, the x coordinate must be even if the ycoordinate is even, or odd if the y-coordinate is odd.
System Architecture
The Delta Design game is a network application, in which
four players who are located on separate computers interact
with each other by way of a shared game surface, a shared
drawing surface, and a chat application. A central server
maintains the current state of all running games. Each player
who logs into one of these games has a pre-assigned role.
The MySQL database stores the following information:
1. The details of the game (start time, duration, agents,
playing surface state)
2. A log of all of the events which take place to modify the
playing board
3. A log of all of the chat communication between the players
4. A snapshot of the state of the game after every action
which modifies the board
5. A record of any events affecting the Delta Design server.
Figure 4a shows the overall system architecture for the
Delta Design online implementation. Every interaction with the
simulation involves sending an event to the server and the
receipt of a response which may contain an arbitrary amount of
3
Copyright © 2013 by ASME
data. This process is illustrated in Figure 4b. Every command
which can be sent to the server is associated with a command
object in the client application. The command object is
responsible for assembling all of the data needed to process the
command on the back end.
Delta Design Game Implementation in Classroom
The authors conducted a series of user studies with this online implementation over a period of 4 years including several
undergraduate capstone design classes. The data collected from
these games (38 four-player teams) provides a rich resource to
explore different aspects of collaborative team practices.
Among the many possible criteria for comparing team
performances, this paper focuses on the time taken for all
conditions to be fulfilled (all green lights). In the case of teams
that never reached this stage, their best score during the game
was used as the metric.
Figure 2. Delta Design canvas mapped on a coordinate system of 150°
Figure 3. Quads constituting deltas in the Delta Design game
4
Copyright © 2013 by ASME
(a)
(b)
Figure 4. a. System architecture and b. event model of the Delta Design game
RESULTS
Table 1 shows the results from the Delta Design game
experiments. The score was computed using the following
rubric: (a) Green light = +1, (b) Yellow light = 0, (c) Red light
= -1. So the highest score possible was +4 (all players have
green lights) and lowest possible score was -4 (all red lights).
The best score included in Table 1 is the highest score that a
given team could achieve throughout the game and the first
time it reached there. For example, some teams with 3 green
lights and 1 yellow light (3x1+ 1x0 = 3) continued to play the
game to get all 4 green lights, but could not achieve their goals;
in such cases the first time the best score of 3 was achieved has
been used in this analysis. The students were not aware of who
had been assigned as other members of their team before the
game started, so all discussions about game strategies and
constraints happened with the clock running.
Note in Table 1 that out of the 38 teams analyzed here,
only 12 teams achieved a score of 3 or more (3 green + 1
yellow lights, or all 4 green lights). Table 2 shows that among
these 12 teams, 50% teams devised a strategy for moving the
tiles on the canvas and discussed beforehand what each
person’s goals were; 16% teams neither strategized nor
discussed their goals beforehand. Since players are provided
with only the primer for their specific role’s responsibilities,
they have to communicate and negotiate with other members of
the team to resolve the conflicting goals.
Following is an example of a conversation students had
about planning a strategy before starting the game:
PM: so part of my spec is that red-blue joints cost a lot
SE: well we want mainly red don’t we
TE: it will be too hot with all red
SE: ohh well thanks thermo man
PM: alright
PM: so we should get everything but my spot green first
SE: ok
PM: I think the order should be structural->thermal->archi>project
TE: well its too cold now
PM: it's a lot of things right now :/
Students also shared information with each other about
their constraints, as in the following conversation:
PM: I think before we begin, everyone should state some
general rules or guidelines that the rest can kind of stick
to, because we have no idea of the constraints of the other
people
Arch: sure, for mine, we can't have external things sticking
around; I need an external perimeter/internal perimeter
ratio of 1.2 I am dealing with the area problem right now
PM: okay. For me, every joint costs money, and joints between
different colors cost more money than joints of the same
color, especially joints along a horizontal line (for
example, the joint between module 5 and 6 right now)
cost 3 times as much
PM: so, just minimize the number of red/blue joints, and
especially along a horizontal line
PM: what about from the other two
TE: I am calculating the temps and need to keep them between
55 and 65. Also too many reds are bad!
SE: I need to make sure that the anchors are not overloaded or
else the structure will fail
Table 2 shows that whether or not the players strategized
before the beginning of the game does not have any marked
effect on the game’s outcome. The same is true for team players
being more transparent about their role’s constraints and
functional requirements. Table 3 shows the Pearson's
Correlation Coefficient (r) values for the user studies. It is clear
that teams which strategize about their game-plan are also
better correlated to share their constraints with each other.
However, this does not seem to make them any more
successful. As the authors continue to examine the data, the
possible reasons for these observations are likely to emerge.
5
Copyright © 2013 by ASME
Table 1. Results from the Delta Design Game Experiments
Team ID
Strategized before game?
Team-7
Team-11
Team-34
Team-35
Team-4
Team-6
Team-8
Team-9
Team-12
Team-14
Team-28
Team-30
Team-1
Team-2
Team-13
Team-15
Team-21
Team-23
Team-26
Team-29
Team-31
Team-32
Team-37
Team-10
Team-17
Team-20
Team-22
Team-24
Team-25
Team-33
Team-3
Team-16
Team-27
Team-36
Team-5
Team-18
Team-19
Yes
Yes
No
No
No
Yes
Yes
Yes
Yes
Yes
No
No
Yes
Yes
Yes
Yes
No
Yes
No
Yes
No
Yes
Yes
No
No
Yes
No
Yes
No
No
Yes
No
No
Yes
No
Yes
Yes
Shared
constraints?
Yes
No
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
No
Yes
Yes
No
No
No
Yes
Yes
No
No
Yes
Yes
Yes
Yes
No
Yes
No
CONCLUSIONS AND FUTURE WORK
This paper demonstrates that the complex rules of the
Delta Design game make it a suitable candidate for analyzing
collaboration strategies in team-based design projects. The
analysis shows that more transparent negotiation tactics, in
which players explain the rationale behind their own tile moves
on the board to other team members and board-moving
strategies, do not improve the teams’ performance. Ideally, we
Time taken
(hours:minutes)
0:58
0:55
1:12
1:15
0:34
0:52
1:12
1:00
0:37
1:08
1:06
0:46
0:41
0:29
1:15
0:44
1:05
1:04
1:10
1:07
0:22
0:17
1:09
0:23
0:59
0:47
0:57
0:17
0:57
1:18
0:38
0:57
0:26
0:13
0:44
0:19
0:01
Best Score
4
4
4
4
3
3
3
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
0
0
0
0
-1
-1
-3
would compare these results with those of equivalent physical
board games. However, our experience with the difficulty in
capturing student interactions and progress during physical
board games is what motivated us to create the online version
of Delta Design in the first place.
We are currently further examining the data collected from
the user studies to analyze the role of politeness between
players, individual players’ dominant behavior and the
relationships between them. We also plan to make a more
6
Copyright © 2013 by ASME
nuanced consideration of how close the teams came to
achieving their goals in lieu of the integers used in this
research, as that can provide a better understanding of which
teams failed completely and why.
ACKNOWLEDGMENTS
This research was supported in part by National Science
Foundation grant 0935127.
REFERENCES
1. Bucciarelli, L.L. “Delta Design Game”, MIT, 1991.
2. Bucciarelli, L.L., “Design Delta Design: Seeing/Seeing
as,” Proceedings of Design Thinking Research Symposium 4,
MIT, 1999.
3. Legardeur, J., Minel, S., and Savoie, E., “A Pedagogical
Game based on Lego Bricks for Collaborative Design Practices
Analysis,” Complex Systems Concurrent Engineering, pp 487494, 2007.
4. Lloyd, P., and van de Poel, I., “Designing Games to
Teach Ethics,” Sci Eng Ethics 14, 433-447, 2008.
5. Grau, M.M., Sheppard, S., and Brunhaver, S.R.,
“Revamping Delta Design for Introductory Solid Mechanics,”
119th ASEE Annual Conference & Exposition 2012, San
Antonio, TX, June 10-13, 2012.
6. Kleinsmann, M., Fleur Deken, Dong, A., and Lauche,
K., “Development of design collaboration skills,” Journal of
Engineering Design, iFirst, pp. 1–21, 2011.
7. Brandt, E., “Designing Exploratory Design Games: A
Framework for Participation in Participatory Design?”
Proceedings of the Ninth Participatory Design Conference
2006, Trento, Italy, August 2006.
8. Besterfield-Sacre, M., Shuman, L.J., Wolfe, H., Clark,
R.M., and Yildirim, P., “Development of a Work Sampling
Methodology for Behavioral Observations: Application to
Teamwork,” Journal of Engineering Education, p p. 347-357,
October 2007.
9. Svihla, V., Petrosino, A., and Diller, K., “Distributed
Expertise and Authenticity in the Development of Design
Expertise,” Proceedings of the International Conference on
Engineering Education (ICEE 2007), Coimbra, Portugal
September 3-7, 2007
.
Table 2. Effect of strategizing and sharing constraints on successful outcomes
Successful Teams
Unsuccessful Teams
Both strategized
before game and
shared constraints
50%
48%
Teams that
Neither strategized
Only strategized
before game or
before game
shared constraints
16%
58%
24%
56%
Only shared
constraints
75%
64%
Table 3. Pearson's Correlation Coefficient (r) for the Delta Design Implementations
Criteria for Correlation
Teams that
Pearson's Correlation Coefficient (r)
Overall
Successful Teams
Unsuccessful Teams
Strategized before game & Shared constraints
0.44
0.29
0.51
(Strategized + Shared constraints) & Best score
0.12
-0.32
0.16
Strategized before game & Best score
0.02
-0.12
0.04
Shared constraints & Time to finish
-0.03
-0.28
-0.02
Shared constraints & Best score
-0.03
-0.41
0.24
(Strategized + Shared constraints) & Time to finish
-0.14
-0.19
-0.17
Strategized before game & Time to finish
-0.2
-0.04
-0.27
7
Copyright © 2013 by ASME