SCHOOL OF INFORMATION TECHNOLOGIES
FIRESTORM: A BRAINSTORMING APPLICATION FOR COLLABORATIVE GROUP
WORK AT TABLETOPS
TECHNICAL REPORT 678
Andrew Clayphan, Anthony Collins, Christopher Ackad, Bob Kummerfeld, Judy Kay
AUGUST, 2011
Firestorm: a brainstorming application for
collaborative group work at tabletops
Andrew Clayphan, Anthony Collins, Christopher Ackad, Bob Kummerfeld, Judy Kay
School of Information Technologies
University of Sydney, Australia, 2006
{andrew.clayphan,anthony.collins,christopher.ackad,bob.kummerfeld,judy.kay}@sydney.edu.au
in a graphical interface (for example, they can be easily rearranged on-screen and saved for later reference), the collaborators are forced to funnel their ideas through a conventional keyboard and screen, which may detract from the
group’s face-to-face discussion. New forms of Single Display Groupware (SDG) [22], such as interactive tabletops,
have the potential to combine the natural, face-to-face discussion supported by conventional brainstorming, with the
increased flexibility that can be gained from computer support. In particular, the generated ideas are easily captured
and a natural gestural interface can be used for discussing
and organising the ideas as a group.
ABSTRACT
The tabletop computer interface has the potential to support
idea generation by a group using the brainstorming technique.
This paper describes the design and implementation of a tabletop brainstorming system. To gain insights into its effectiveness, we conducted a user study which compared our system
against a more conventional approach. We analysed the processes and results with the goal of gaining an understanding
of the ways a tabletop brainstorming system can support the
phases of this activity. We found that our tabletop interface
facilitated the creation of more ideas and participants tended
to create more categories. We observed that the tabletop provides a useful record of the group processes and this is valuable for reviewing how well a group followed recommended
brainstorming processes. Our contributions are a new tabletop brainstorming system and insights into the nature of the
benefits a tabletop affords for brainstorming and for capturing the processes employed by a group.
Conventional brainstorming consists of two phases: an idea
generation (storming) phase, and an idea categorisation (norming) phase. The sole purpose of the storming phase is to
elicit as many possible ideas from the group members so as
to create a checklist which can be later used as the basis for
the problem solution [16]. To be effective, it relies on four
principles: (1) criticism is not permitted (adverse judgement
of ideas must be withheld until later); (2) unusual ideas are
welcomed (the wilder the idea, the better: it is easier to tone
down than to tone up); (3) quantity is encouraged (the greater
the number of ideas, the increased likelihood of useful ideas);
and (4) combination and improvement are sought (participants should suggest how ideas of others can be turned into
better ideas or how two or more ideas can be joined into another idea). Following this phase, the norming phase is employed where the generated ideas are discussed, evaluated,
merged and any duplicates are removed.
H5.2 [Information interfaces and presentation]: User Interfaces - Graphical User Interfaces
ACM Classification:
General terms:
Design, Human Factors
Keywords: Tabletop interface, Surface computing, UserCentered Design, Group work, Brainstorming
INTRODUCTION
Brainstorming has been used effectively in a range of group
activities as a technique to enhance creativity and articulate
ideas [16]. Brainstorming helps a group of people generate original ideas, with new ideas inspired from older ones.
It encourages egalitarian participation and enhances collaboration within groups. It is a critical step in the design
process [1], prompting its wide-spread use in many contexts. While brainstorming predominately takes place without the use of computers, on paper or whiteboard, there are
several computer-based systems that support brainstorming
[6, 21, 23]. Though these have several advantages due to
the fact that the generated ideas are digitised and represented
There are some key problems that can reduce the effectiveness of brainstorming. Diehl and Stroebe [4] identified three
of these: (1) free riding; (2) evaluation apprehension; and (3)
production blocking. Free riding occurs when an individual
believes their contribution is dispensable, where this may in
part be due to the size of the group, with a larger group giving a greater sense of anonymity. Evaluation apprehension
is the fear of negative evaluations from other group members; this causes participants to restrain themselves from expressing ideas. Production blocking is caused by the delay
in verbalising an idea as it occurs. By not releasing the idea,
it is likely to be forgotten or suppressed due to the feeling
that the idea is no longer relevant or original at a later period.
Mullen, Johnson and Salas [15] also report similar findings.
Of the three problems, Diehl stated that production blocking was shown to the largest detractor from success. This
was also confirmed in studies by Lamm and Trommsdorff
1
contributions and noting future work.
[12] which showed the issue of a single person talking at
a time had an effect and the studies by Bouchard and Hare
[2], who commented that a longer delay between thought and
announcement raised issues. These factors are important to
consider when exploring new ways to support brainstorming
by collocated groups at a tabletop.
RELATED WORK
We build on the large body of work studying the problems
associated with brainstorming and the many systems for supporting it. Electronic support for brainstorming has been
of two forms: for conventional desktops, supporting brainstorming in a collocated or distributed setting; or for SDG
such as tabletops, for supporting a group of collaborators in
a collocated setting. We now describe key research in each
of these areas.
With the recent availability of affordable tabletop hardware,
it is timely to explore and study the design of new software applications that will support collaborative group work.
Given the value of brainstorming in many contexts, it is important to understand how a tabletop interface can be designed to support this process, while at the same time considering what impact the nature of a tabletop interface will have
on the group processes and outcome. There are several important aspects that need careful consideration. The first is to
consider the means for entering ideas in the storming phase.
This must be designed to minimise production blocking and
to increase the quantity of ideas generated. This is challenging for tabletops, where text entry is often problematic
with on-screen keyboards [9], and writing with a pen/stylus
is slow. Second, the technology should not be distracting.
For example, this could occur if large amounts of interaction
are required in the process of entering ideas (for example,
gesturing to create a new note, or moving the previous notes
out of the way to make room for others). Third, the interface
should be designed to support a small group of users working around the tabletop at different orientations so that they
are facing each other and can easily hear the ideas called out.
Finally, the interface should foster awareness of the group
members’ actions, where the collaborators can glance at the
tabletop to quickly gain an overview of the pool of ideas being generated.
Problems associated with brainstorming
As described in the introduction, several researchers have
studied the problems that commonly hinder the performance
of groups when brainstorming [2, 4, 12, 15]. In response to
these issues, a number of techniques and systems that build
on the traditional group brainstorming method have been formulated. These include: nominal brainstorming, where each
participant brainstorms in isolation at the storming phase,
and then the group comes together for the norming phase
[4]; a group passing technique [24], where an idea is written
down, passed onto another person, who adds to the received
idea and passes the sheet along; and question brainstorming
[18], where a list of questions is brought prior to the session
to stimulate the session itself. Other techniques diverged into
the area of concept mapping as a process to brainstorming,
utilising associations to create ideas.
To advance brainstorming, Diehl [4] suggested — as a solution to the backlog problem — to allow group members to
write their own ideas down as opposed to the original single scribe. This has become common. Furthermore, the introduction of post-it notes allows ideas to be written such
that they can be easily repositioned (for example, on a whiteboard, in contrast to the conventional approach where ideas
are written down directly on a large piece of paper or whiteboard [16]). The post-its also aid in making the categorisation of ideas easier in the norming phase.
To explore the interface mechanisms required for brainstorming, we designed and built a tabletop interface that takes
account of these issues. We designed it taking careful account of the key considerations listed above, reducing any
blocks to productivity [4]. Our system consists of a large
multi-touch tabletop, where each user has a physical wireless
keyboard. This facilitates efficient text entry. At the same
time, it discourages users from interacting with the tabletop
display during the storming phase, to minimise distractions
from generating ideas. The interface is designed to support
egalitarian interaction, not favouring any particular orientation. To understand what impact the tabletop interface has on
the processes and final outcome, we conducted a study which
compared its use against brainstorming at a non-interactive
whiteboard. This is to assess whether the interface has any
impact on the idea generation or organisation phases, and
whether the tabletop interface retains the benefits of face-toface discussion.
Electronic brainstorming using computers
As technology became available, brainstorming moved to
new media making use of computer networks to express
ideas [6, 21, 23]. Electronic brainstorming addresses several of the problems associated with traditional brainstorming — by parallelising the input, reducing apprehension [3],
and minimising free riding. A survey of such systems can
be found in the work by Jessup and Valacich [11]. However,
performing brainstorming at a computer can reduce face-toface interaction, which has been highlighted as particularly
important for team building and problem solving [5]. This
has inspired researchers to explore new forms of SDG for
supporting brainstorming, which we describe next.
Electronic brainstorming at a tabletop
The remainder of this paper is organised as follows: the next
section explores related work on the nature of brainstorming
and conventional computer interfaces for brainstorming. We
then describe the design of the system, which aims to avoid
the common problems encountered in brainstorming. After
detailing the design of our evaluation, comparing brainstorming at a tabletop to a conventional whiteboard, we report and
discuss our results. We then conclude by reflecting on our
Several forms of new technology have been explored for the
storming and norming phases. Interactive digital wall displays, for example, allowed a similar concept to a whiteboard, but with the ability to group and move items easily
[7]. Several researchers have explored using tabletops as
interfaces that are somewhat relevant to brainstorming, although none of them aims to support the classic form of the
2
nication, even though the members are working at different
positions at the tabletop. We also aim to improve the brainstorming process compared to conventional methods, such as
at a whiteboard or on paper. We now briefly describe each of
our key design goals.
techniques for idea generation. Three notable systems that
have informed our design are Hunter and Maes’ Wordplay
[10], Bao et al.’s Momentum [1] and Hilliges et al.’s system
for collaborative creative problem solving [8].
Wordplay [10] aimed to provide a multi-touch platform that
supports collaborative discussions. To make an idea in this
system, a participant speaks into a microphone or selects a
multi-touch on-screen keyboard. It supports a brainstorming
method that is a variation of the original, with a facilitator
preparing a table with a suitable background. The system
links with semantic knowledge databases, allowing a novel
association feature to aid idea generation. It also uses animation for emphasis of ideas (for example, an idea fades away
if not touched within two minutes).
DG1: Support fast, concurrent idea generation
It is important to design the interface so that users can generate a large number of ideas, and to minimise production
blocking [4]. It was critical to select an appropriate mechanism for idea entry [9]. We concluded that a physical keyboard was the best choice as it supports quick and accurate
text entry, for the large proportion of users familiar with keyboards. We did consider several other options. If the tabletop is stylus-based, users could write ideas. However, this is
slower than typing and may also be tiring if the users need
to write many ideas. If the tabletop is touch-based, users
could write with a finger. But this is not particularly natural or quick and is likely to disrupt idea generation. We also
consider on-screen (virtual) keyboards. We rejected these as
they are somewhat difficult to use at tabletops, mostly due to
a lack of tactile feedback. This too is likely to slow people
down in the storming phase.
Momentum [1] has a focus on the step before a brainstorming session. In this, it is a creativity support tool to encourage
some individual preparation, which is later used as stimuli to
the main brainstorming session. This work is motivated by
Pinsonneault et al. [17] and Wang et al. [25] on the importance of seeding ideas in brainstorming. Thus, their method
is quite a modification to classical brainstorming, and indeed
combines some form of nominal element before the group
convenes.
For the design goal of fast and concurrent idea generation, we
designed our system with physical keyboards for idea input.
Each user is provided with a low-profile wireless keyboard
(with full-sized keys), that they can place on the rim of the
tabletop. The keyboards can be easily moved around so as to
not force the users to remain at a particular position. Another
important advantage of the keyboard is that it supports the
storming phase — it makes it natural for users to keep their
hands off the table and so, it reduces the temptation to start
parts of the norming phase. This also means that a group of
users can work even on a relatively small tabletop, as there
is no need for each user to have a region of the screen for
typing or writing their ideas.
Finally, Hilliges et al. [8] studied collaborative creative problem solving in an environment with both a tabletop and interactive wall display. The tabletop is used for input whilst
the wall is used as a peripheral display for gaining awareness
of the overall structure and content of the ideas generated.
This is particularly important as their system has collaborators working on opposite sides of the tabletop, with their own
personal territories. Their study involved a brainstorming
technique based off the brainwriting technique of VanGundy
[24]. In this method, group members write their ideas on a
piece of paper that is then placed in the center of the table
for another member to read prior to writing their next comment. This is a serious departure from the essential nature
of the storming phase. Their evaluation compared the use of
the tabletop and whiteboard with a paper-based method, that
involved writing post-it notes and organising these on the
whiteboard. They report that the participants preferred the
electronic system (to the paper-based whiteboard) in 80% of
cases, and that the tabletop resulted in lower output of ideas
(though not statistically significant).
Since the user needs feedback on the text as they type it,
we provide this in a small post-it-like element. This can be
moved to the position that a user finds convenient. This aspect of the design ensures that users face the table and so, as
they call out their ideas, they can hear each other. When the
user has finished typing an idea, they press the return key.
This causes that note to go to the collection of ideas created
so far. Their post-it becomes blank and is ready for their next
idea.
We wanted to go beyond this work and create a system that
stays close to the traditional brainstorming method [16, 4],
and that is designed to take account of the constraints and affordances of tabletops. We aim to support groups of around
four people as this number can be conveniently accommodated around available tabletops and is an effective number
for brainstorming. We next describe the design goals behind
our approach.
DG2: Many ideas displayed on the tabletop at once
If the tabletop interface is to effectively support multiple
users, it is important that many ideas can be represented on
the tabletop at once. It is also important to ensure the text
is sufficiently large and legible at all times, so that the group
can easily see the current state of the brainstorm. This means
the representation of the ideas on the tabletop needs to be
designed carefully to minimise the space occupied by each
idea, while ensuring all ideas are legible.
DESIGN GOALS FOR TABLETOP BRAINSTORMING
We aimed to design a brainstorming interface that effectively
supports a small group of users working around a tabletop.
These design goals are grounded on the constraints and opportunities imposed by tabletop interaction. In particular, we
aim to ensure a high degree of group awareness and commu-
DG3: Don’t enforce orientations or territories
The interface should enable the users to work from whichever
position they prefer at the tabletop; the interface objects (and
physical keyboards) should be moveable. While personal
3
working territories are naturally formed at a tabletop [20],
we want people to be able to decide where they prefer to be
and to be able to easily move if they wish. Once an idea has
been created, we release it to a shared area, in clear view for
all group members. This means the interface should be designed such that the collaborators are free to touch any particular ideas on the tabletop, without feeling like they are
invading another user’s personal working area.
DG4: Aggregate a pool of ideas in the middle of the table
A challenge in designing tabletop interfaces is effectively
displaying information to support people working from multiple orientations [19], as needed for supporting a small group
working face-to-face. Furthermore, the interface should ensure group awareness during the idea generation phase, in
order to allow the collaborators to gain an overview of the
ideas to inspire new ones. We chose to present the generated
ideas in a radial layout in the centre of the tabletop. This
avoids favouring any particular orientation or area of the table. All ideas are treated equally regardless of who added
them. For each user, many of the notes will face their orientation; findings in [14] suggest that short pieces of text that
are facing away from users may still be read easily.
Figure 1: Four collaborators using Firestorm to organise their ideas in a brainstorming task.
process to see how it evolved, in order to give them feedback.
This capturing of state would also allow the group to jump
back to previous points (for example, to restore an idea that
was previously discarded). Therefore, a key design goal is to
capture as much information during the brainstorm to exploit
it in future work.
DG5: Code each idea to show the user who created it
FIRESTORM: USER VIEW
We colour code the ideas that each person creates. This design decision involved a compromise. We were strongly influenced by the need to discourage free riding. Since this is
an acknowledged problem for effective brainstorming, it is
useful to make it easy for each person to be aware that their
output will be discernible. The tradeoff that we considered
was that there are some benefits in anonymity since it may
help an individual feel freer to generate wilder ideas. This
benefit seems rather small, given that the group members are
actually at the table together and can see and hear the ideas
being generated. They are likely to notice who calls out a
very wild idea.
Based on these design goals, we created Firestorm (Figure
1). Firestorm was built and refined in two stages. We implemented an initial design and evaluated it in a study involving
12 participants, in four groups of three. They used both the
interface and a more conventional approach at a whiteboard.
We used this to refine our interface. We now describe the
interface in terms of each of its primitives for creating and
organising ideas on the tabletop.
Creating ideas in the storming phase
At the beginning of the brainstorming session, Firestorm
presents one text input note for each keyboard. Figure 2
shows the case of four users, each with a keyboard and these
users have placed the associated feedback notes in front of
them, at the corners of the table. When a user types on his
keyboard, the text appears immediately on his corresponding note. Pressing return on the keyboard adds an idea to the
table; each new idea is displayed as a new note in a spiral
in the middle of the tabletop (as shown in Figure 2). This
avoids favouring any particular orientation, and it enables all
collaborators to gain an overview of the ideas generated so
far. After adding a new idea to the middle, the original note
(linked to the physical keyboard) is cleared and remains in
the same location on the tabletop. Notably, the process of
entering notes does not require the user to perform any action at the tabletop. The layout happens automatically, so the
collaborators can remain focused on brainstorming ideas and
entering them with the keyboards.
DG6: Support flexible grouping for idea convergence
With the ideas placed in a pool in the middle of the table, it
is important that the interface provides flexible mechanisms
for discussing and grouping the ideas for convergence (in the
norming phase of the brainstorm). Thus, the users need a way
of quickly sorting the new ideas. This may come in the form
of gestures to support the selection and grouping of multiple
items at once. It may also involve investigating alternative
metaphors for representing collections of topics (e.g. in the
form of piles [13]). Another consideration is how the ideas
can be structured. A design goal is to provide a hierarchical interface, where multiple levels of nesting are allowed,
enabling collections of related ideas that can be manipulated
all together with one interface action (for example, a whole
part of the tree can be moved by dragging the parent).
DG7: Capture the group’s process and final outcome
In an electronic brainstorming system, there is the ability to
capture each stage of the brainstorm, so that it can be saved
and reviewed later. This can be useful in many contexts for a
tabletop, to enable an ‘outsider’ to see how the group’s ideas
evolved. For example, in a learning context, a teacher —
who would be unable to monitor all the groups at once —
is able to walk around to each group and replay the group’s
The appearance of each note was refined based on our findings in the initial usability studies. Originally, each note was
designed to look like a physical post-it note, that allowed
multi-line text entry. However, we discovered that the participants tended to utilise only a small portion of the available space on the note for text. This lead to a redesign of
the note from a standard post-it square to a rectangular strip,
4
Figure 2: Firestorm at the end of the storming stage.
Notes are shown in a spiral in the middle of the table; the notes in the corners are linked to the physical
keyboards.
Figure 3: Firestorm at the end of the norming phase.
Containers are made by flipping a note; in the figure
there are four containers/categories, each with a title.
thereby reducing the space it occupied on the tabletop. At
the same time, the size of the text was increased, with the
text gradually reducing in size to ensure all text could fit on
the note. For a typical brainstorm, each note has at most a
few words; so the text will remain large and clearly legible
on the tabletop.
Given that the users will need to organise many notes at a
time on the tabletop, we incorporate a lasso gesture to simplify working with a group of objects at once. This was designed to address feedback from our initial usability study,
where participants wanted a way to easily move several ideas
that were in one area of the table. This happens quite often
as the users realise that two sets of ideas can successfully be
combined.
The interface was also enhanced to colour each note according to the author. In paper-based brainstorming, the handwriting on notes provides a subtle indicator of who wrote it.
The colouring of notes on the table enables a similar form
of ownership, while avoiding extra text on the note indicating the author, which could create clutter in the interface.
The colouring of the notes will also enable the collaborators
to compare the portion of notes that they generated with the
contributions of the group. As noted previously, this may
help deter users from free riding.
To use the lasso, the user drags their finger over multiple
notes, then moves their finger to the location they would like
to move them to (as shown in Figure 4). After holding their
finger still for one second, the objects move to the location
of the user’s finger, arranged in a grid. This enables a user to
select a whole group of items and move them to a particular
location. The user can also use the lasso to move a group of
notes into a container (as shown in Figure 5).
Throughout the storming and norming phases, all interface
actions are saved. This enables the brainstorming session
to be replayed from the beginning, in order to achieve our
design goal of capturing the process for later analysis and
reflection.
Evaluating and grouping ideas in the norming phase
After generating ideas with the physical keyboards in the
storming phase, the collaborators can evaluate and group the
ideas with the provided manipulation tools. All objects in
the interface can be moved, rotated and resized. In addition,
there is a flip gesture that enables a note to become a container. We envisaged that this would be useful for grouping
ideas by placing them on the back of the note with a higher
level idea. A flip is initiated by selecting a note from within
the middle of one of its edges (the selection areas are highlighted when the user is touching the note). Once flipped,
the text of the note is displayed above, making it easy for
other ideas to be placed into the flipped note (now acting as
a container). Containers can also be nested to create deep
hierarchies of ideas. Figure 3 has four notes that have been
flipped to provide containers for grouping ideas. The ideas
can be arranged within the containers, and they can be easily
dragged-out. The container can also be enlarged and rotated,
which also transforms the notes contained within it.
EVALUATION
The primary goal of our evaluation was to assess the effectiveness of our design and resulting tabletop interface for
supporting the task of brainstorming. To do this, we chose
to compare its use against that of a conventional whiteboard,
to study the differences in processes and the effect on the
final outcome. The study was mostly qualitative, to observe how the groups used each experiment condition, and to
gather feedback on participants’ perceptions on which condition was more natural to use and which produced the better
results. At the same time, we performed a quantitative analysis to compare the number of ideas generated (in the storming
phase) and their evaluation and categorisation (in the norming phase).
The recycle bin, shown in the middle of Figure 3, is a container that is always visible and is used to discard any ideas
that are no longer wanted (by dragging them into it). It can be
moved to any location on the tabletop. Like regular containers, the users can also drag notes out if they want particular
notes to be restored.
Experimental Design
The experiments had a within-subject design, and were double cross-over; each group completed a brainstorming task
using both the tabletop and whiteboard conditions, and the
ordering was balanced to counter any learning effects. The
5
The brainstorming task had some specific instructions about
how to conduct the brainstorming session. The brainstorming had the conventional storming and normal phases, and
the participants were instructed to do each phase for 10 minutes on the specific topic. We instructed participants to do
this to ensure they approached the task in the same manner
for both conditions, to ensure consistency in the results. With
the tabletop condition, this involved producing as many ideas
as possible in the first 10 minutes, and then collaboratively
evaluating and organising the ideas in the remaining 10 minutes. With the whiteboard condition, after the idea generation (storming) phase, the participants were given a print out
of all the written ideas, so that they could erase the board
to make the assessment and categorisation (norming) phase
more flexible.
Figure 4: Lasso gesture to re-orient notes
on the surface. (a)
original, (b) notes selected with finger, finger now over background, (c) notes reoriented.
After the participants had finished the tasks, they completed
a post-experiment questionnaire. This consisted of 7 Likertscale questions that asked participants about their experiences with the tasks, scenarios and interfaces. Each question was asked for both the tabletop and whiteboard conditions; the Likert-scales were arranged side-by-side for the
two conditions, and space was provided for a free-form response. Following, four questions elicited general qualitative feedback on the two interfaces in the form of commentary responses. This was followed by a short semi-structured
interview with 9 questions, that enabled the experimenter to
clarify any observed phenomena.
Figure 5: Lasso gesture to move notes
into a container. (a)
original, (b) notes selected with finger, finger now over container, (c) notes attached.
Apparatus
The tabletop used in the experiment was an 46-inch LCD display, which had a touch overlay capable of sensing up to 32
simultaneous touch points. The display had a resolution of
1920x1080 pixels, which ensured the text entered was legible even at small sizes. As the tabletop had a wide-screen
aspect ratio, it was comfortable for the participants to stand
with a partner on each of the long sides of the tabletop, although they were free to position themselves as they wished.
Each participant was provided with a low-profile wireless
keyboard (with full-sized keys), which could be moved freely
around the tabletop display and placed on the rim when typing.
groups completed a different task with each condition, although the task required the same forms of information, but
for a different problem domain. Task A had participants
brainstorm about ways to promote their country to increase
tourism. Task B had participants brainstorm about ways to
promote their university or workplace for stimulating recruitment. The task assigned to each condition was also varied in
the experiments.
Each group in the experiment had four participants. This is a
plausible number for a group brainstorming session, enabling
a potentially large number of ideas to be generated in the one
session. It is also a fair size for both the tabletop and the
whiteboard, based on the hardware sizes available for the experiment (the apparatus is described in a later section) — the
tabletop could be used comfortably with two users on each
of the longer sides of the table, and the whiteboard was large
enough to accommodate four users writing ideas simultaneously. The study was designed carefully to not favour any
particular condition.
The whiteboard, shown in Figure 6, was 65-inches and was
electronic but not interactive. A printer attached to the whiteboard enabled the contents to be quickly captured and printed
to a sheet of paper. Each participant was given their own
marker and there were several additional markers (of different colours) that were placed below the whiteboard.
The experiment was video and audio recorded to enable later
analysis. An experimenter was present who gave initial instructions and then retired to a separate observation room
whilst the groups worked.
The experiment procedure began with a background questionnaire, to gain general information about the participants’
computer skills and past brainstorming experience. We deemed
this information important to assist in our analysis of any interesting behaviours observed in the experiments. Following this, the participants were each given a task sheet that
explained the problem they needed to brainstorm about —
either promoting tourism for their country or driving recruitment for their university/workplace.
Participants
The evaluation had 24 participants (18 male, 6 female) organised into 6 groups, with an age range of 20-54 (mean
26). Each group consisted of people who knew each other
beforehand to ensure the task did not make them additionally
uncomfortable. The participants were predominately university students studying IT-related degrees. 11 participants had
6
Order
(Task)
G1 W (A)
T (B)
G2 T (B)
W (A)
G3 W
T
G4 T
W
(B)
(A)
(A)
(B)
G5 W (A)
T (B)
Figure 6: The whiteboard used in our evaluation, at
the end of a norming phase.
G6 T (A)
W (B)
other backgrounds, encompassing architecture, engineering,
advertising and business. Based on the background questionnaire responses, all participants used computers at least
11 hours per week, and all were rated themselves as being
competent or above with a computer. 18 had used a tabletop
before, but for unrelated experiments. All participants had
used a whiteboard before.
Summary
Used a concept mapping technique for generating
ideas on the whiteboard. For the tabletop, the group
required agreement on any ideas before they added
them to the tabletop (against instructions provided).
All participants wrote down ideas (on both interfaces). This came about due to one member pointing out the other members could all write (whiteboard) and type (tabletop) at the same time.
Used a mind-map approach to create ideas on the
whiteboard and organisation through associations.
Exhibited territorial behaviour, in both interfaces;
each person had their own space, and moving into
someone’s space was avoided.
The whiteboard had one scribe, similarly the tabletop had one main person entering ideas. They
touched the table in the storming phase (against instructions provided). Members were shy.
Group had a single scribe on the whiteboard. Very
reliant on the scribe’s memory to hear others’ ideas
and transcribe them. On the tabletop, all participants entered ideas concurrently.
Table 1: Summary of group actions on the tabletop and
whiteboard brainstorming interfaces. The ordering of
conditions and topics is indicated in the second column
— W(A) means Whiteboard with Task A; T(B) means
Tabletop with Task B.
RESULTS
Each group exhibited a number of approaches, strategies and
styles towards brainstorming in both the idea organisation
and categorisation phases. Table 1 shows the condition and
task ordering of each group, and summarises their key characteristics. We now elaborate on the task performance of
each group.
became one of the two main scribes. This led them to discussion, but also allowed for the generation of ideas, through associativity. At the tabletop, their strategy was similar, but the
self-imposed bottleneck from the whiteboard interface was
removed, as all participants entered ideas concurrently.
Group 4 was similar to Group 2, in that they split the whiteboard into 4 columns. The main difference arose in the
norming phase at the tabletop. This group — while aware
of each other’s ideas — were territorial for most of the session. When a member moved into another’s assumed area,
it often got in the way of what that individual was doing. It
was only towards the end of the norming phase that collaboration started to take place (along the widths of the table, in
pairs). Notably, they were the only group that did not finish
the norming phase after the 10 minute allotment, as their organisation process was slow due to the visible territoriality
(the participants avoided touching others’ ideas for much of
the session). The interview revealed that there were strong
personalities on each side of the table, which may have contributed to this.
Group 1 consisted entirely of students from a design/architecture background. They utilised a concept mapping technique to brainstorm on both interfaces. They reported they
had learnt this as part of classes they took in the past. Due
to this, the storming phase was composed of lively discussion, but a low degree of idea generation. At the start of the
norming phase they used symbols on the whiteboard to denote objects that fell in multiple categories. On the tabletop,
due to the small number of ideas (show in Figure 7), they finished categorising in three minutes, in part due to the flexible
primitives such as note containers.
Group 2 consisted of an even split of IT-related PhD students
and researchers. They followed the brainstorming instructions and generated ideas. In the storming phases (for both
conditions) they generated and verbalised their ideas while
working at a rapid pace. Notably, they devised personal
territories on the whiteboard so that they could write ideas
concurrently. In the norming phase (for both conditions),
they discussed which ideas were similar and grouped them,
through a combination of an initial individual placement and
then group agreement for ideas that were not placed. Overall,
this group adopted a similar strategy for both conditions.
Group 5 assigned a single scribe for the whiteboard and also
had only one main contributor on the tabletop. They exhibited shyness, even though the participants knew each other.
They were the only group to touch the tabletop (to move
ideas around) in the storming phase. Their preoccupation in
categorising ideas during the storming phase (instead of deferring this to the norming phase) may explain the relatively
low number of ideas created (see Figure 7).
Group 3 used a mind-map approach on the whiteboard to
generate ideas. This was sparked by one participant, who
Group 6 initially decided to each write with a different coloured
7
Figure 7: The number of ideas generated at the end
of the storming phase (after 10 minutes) on each interface. Shown with a white line in the middle, is the
session at 5 minutes in. Total ideas are shown above
the columns.
Figure 8: The number of ideas generated in the storming phase with each condition.
marker, but chose to have a single scribe after group member
6B suggested “we could just get one person to write.” 6A
remarked in the interview “we were dependent on the scribe,
hoping he heard”. This was supported by the video analysis, where the scribe was often seen repeating ideas out loud,
to re-jog his memory. On the tabletop, all users entered information with their keyboard after 6B suggested “I think
maybe we should just generate the ideas first, then we can
organise it after, just type as much as possible for now.” The
team members remarked that the tabletop made ideas “flow
easier” and they were quicker to enter. Participant 6A stated
“the tabletop felt more collaborative, while the whiteboard
was limited and only had one person as scribe.”
Figure 9: The number of categories created by each
group in the norming phase. The white numbers at
the bottom represent the average number of ideas per
category.
Figure 7 shows the number of ideas generated by each group
(for each condition) in the storming phase. The chart indicates that the number of ideas generated at the tabletop was
equal or greater than the whiteboard in 4 of the trials, with the
exception of groups 1 and 5 who did not write ideas concurrently at the tabletop (see Table 1). Figure 8 shows the number of ideas generated on a per minute basis. Four groups
generated ideas at a faster rate using the tabletop (Groups 2,
3, 4 and 6).
to the Likert-scale questions, which were asked of both conditions. The results show that in most cases the participants
found the two conditions mostly similar. The results of a
Wilcoxon signed-rank test indicate that Question 5 (understood how the system responded to input) and Question 7
(able to enter information concurrently) are statistically significant (p = 0.006 and p = 0.003, respectively) at the 10%,
5% and 1% levels. Question 5 favoured the whiteboard due
to the participants’ previous use of whiteboards, and Question 7 was due to the physical keyboards (when using the
tabletop) and the tendency for some groups to assign a scribe
(when using the whiteboard).
Figure 9 shows the number of categories for each group at
the end of the norming phase, and also the average number of
ideas in each category (shown at the bottom of each column).
This indicates that the number of categories was similar for
both conditions, with the exception of groups 3 and 4. Group
3 had a large number of categories due to the relatively high
number of ideas generated at the tabletop (see Figure 7), although the size of the categories was much the same as with
the whiteboard. Group 5 had a larger number of smaller categories on the tabletop (compared with the whiteboard). This
was in part due to the group touching the table and organising
the ideas in the storming phase, so when the norming phase
formally started they already had a number of categories.
Responses to the free-form questions indicated that the whiteboard had crowding problems, which encouraged the assignment of scribes. For this reason, the participants commented positively on the tabletop for enabling all four of
the group members to work simultaneously. For example,
2C stated “we all had uninterrupted access to the tabletop;
the whiteboard had crowding issues at times.” Similarly,
3A stated “the tabletop had a better layout for concurrent
use, we could stand around it, facing one another; we could
also contribute independently to start with.” 5C commented
We now report on the participants’ responses to the postexperiment questionnaire. Table 2 summarises the responses
8
Question
1 Able to represent my ideas about
the topic.
2 Able to easily understand the core
operations presented to me.
3 Found the system made it easy to
co-ordinate with other members
of my group to do each of the
tasks.
4 Found it easy to use the system to
do each of the tasks.
5 Able to understand how the system responded to my input.
6 Able to enter information into the
system easily.
7 Able to enter information concurrently.
We observed groups 1 and 5 in particular adopting the scribe
approach as they thought that is how to best conduct brainstorming at a whiteboard, based on their previous experiences. This is despite the task sheet asking the users to
all write their ideas concurrently. The second contributing
factor is the static nature of the whiteboard; by assigning a
scribe, the participants avoided any crowding at the whiteboard. Groups that did not assign scribes created more ideas
(the scribe role produced a ‘bottleneck’ in the idea generation phase), but comments did indicate that crowding was
one disadvantage of using the whiteboard for brainstorming
in a group of four.
Mean (SD)
WB
TT
6.08 (.83)
5.96 (1.08)
6.58 (.72)
6.46 (.66)
5.83 (1.01)
6 (.88)
5.96 (.81)
6.08 (.83)
6.67 (.7)
6.13 (.8)
6.17 (.82)
6.42 (.65)
4.96 (1.73)
6.5 (.72)
By contrast, the results do not indicate that crowding of people at the tabletop was problematic. Our design goals explicitly accounted for this, where we pooled the generated
ideas in the middle of the screen (DG4) to avoid influencing territoriality (DG3). This meant that the four users could
work concurrently in the idea generation phase, resulting in a
high number of ideas being generated compared to using the
whiteboard. Notably, the participants favoured the tabletop
in terms of concurrent idea generation (as indicated by the
post-experiment questionnaire results). The issue of territoriality seems an important consideration in the design of tabletop brainstorming interfaces; physical keyboards are one way
of avoiding the need for users to form territories while entering text (using either stylus or touch) during the storming
phase.
Table 2: Likert-scale responses from the postexperiment questionnaire summarised by median (7
being strongest agreement; the median is the middle
value of the sorted list of values for a question). The
results are listed for the whiteboard, then the tabletop.
Those shown in bold represent those that are statistically significant.
on the limited workspace awareness imposed by the whiteboard, stating that “each participant writing their ideas on
the whiteboard simultaneously was not practical as duplication of ideas would have occurred.” Finally, the physical keyboards and appearance of notes on the tabletop was also well
received. For example, 6A stated that the “tabletop interface
was easier to read and write on, creating more time to brainstorm”.
The results also indicate that the tabletop performed effectively at supporting the norming phase of the brainstorming.
The provision of a lasso gesture (as part of our DG6 for flexible grouping) was positively received for the task of selecting
a several notes and placing them into containers. Future work
will also augment the lasso to present a pop-up menu that allows the users to cycle through different sorting methods for
the ideas; this is to support group discussion of a set of ideas
in the middle of the table.
DISCUSSION
We set out to design a tabletop interface for brainstorming,
while ensuring that it supports a small group of users effectively in the storming and norming phases. The experiment results indicate that the design of the tabletop interface
resulted in a comparable or — in four of the six trials —
greater number of ideas generated with the tabletop (compared with a non-interactive whiteboard). A key factor contributing to the results for efficient idea generation (Design
Goal 1, or DG1) is the mechanism for entering the ideas. To
achieve DG1, we chose physical keyboards. We found that
the physical keyboards were appreciated by the participants,
with some favouring the use of a keyboard over handwriting,
both in terms of speed and reducing fatigue. In addition, we
found that the appearance of the idea ‘notes’ on the tabletop
was effective (DG2,5); there were no complaints of clutter
and the readability of text was praised
CONCLUSIONS
Inspired by the potential of tabletops for small group collaborative work, we set out to design a tabletop interface for
enabling the common technique of brainstorming. To take
account of the common problems in brainstorming [4], we
formulated a set of design goals which informed the design
of the interface facilities for storming and norming. We then
studied the new interface by comparing its use for a realistic
brainstorming task with a conventional whiteboard.
While not explored in this study, the users’ interactions at the
tabletop were all captured (DG7); this allows us to ‘replay’
the whole brainstorming process, which we will exploit in future work. For example, enabling participants to go back and
undo actions, or allowing the whole brainstorming process
to be replayed for reflection (in terms of individual contributions) or assessment by a teacher (which relies on DG5).
However, there are some notable effects that the device (tabletop or whiteboard) had on the way that the participants approached the task (thus affecting the final outcome). The
whiteboard tended to encourage the assignment of scribe
roles, due to two reasons. First, all participants had conducted brainstorming in the past, and this likely would have
influenced the way they approached the task at whiteboard.
The results of the study highlight that several aspects of our
design goals were effective. In particular, we found that our
approach of using physical keyboards, avoiding territoriality
by pooling ideas in the middle of the table, and providing a
simple mechanism for creating categories, led to positive re9
11. L. Jessup and J. Valacich. Group support systems: New
perspectives. Prentice Hall Professional Technical Reference, 1992.
sults in terms of the number of ideas generated and the feedback on using the tabletop for brainstorming. Given the unfamiliarity of the new tabletop interface, the results represent
a clear indication of learnability and ease of use. This work
contributes the interface design goals, and an understanding
of the effect of the different design choices on group brainstorming at a tabletop.
12. H. Lamm and G. Trommsdorff. Group versus individual performance on tasks requiring ideational proficiency (brainstorming): A review. European journal
of social psychology, 3(4):361–388, 1973.
ACKNOWLEDGEMENTS
13. R. Mander, G. Salomon, and Y. Y. Wong. A “pile”
metaphor for supporting casual organization of information. In Proc. CHI ’92, pages 627–634, New York,
NY, USA, 1992. ACM.
We gratefully thank our experiment participants for their
time and helpful feedback. This work is partly funded by
the Smart Services CRC.
REFERENCES
14. M. R. Morris, A. Paepcke, T. Winograd, and J. Stamberger. TeamTag: exploring centralized versus replicated controls for co-located tabletop groupware. In
Proc. CHI ’06, pages 1273–1282, New York, NY, USA,
2006. ACM Press.
1. P. Bao, E. Gerber, D. Gergle, and D. Hoffman. Momentum: getting and staying on topic during a brainstorm.
In Proceedings of the 28th international conference on
Human factors in computing systems, CHI ’10, pages
1233–1236, New York, NY, USA, 2010. ACM.
2. T. Bouchard Jr and M. Hare. Size, performance, and
potential in brainstorming groups. Journal of applied
Psychology, 54(1p1):51, 1970.
15. B. Mullen, C. Johnson, and E. Salas. Productivity
loss in brainstorming groups: A meta-analytic integration. Basic and Applied Social Psychology, 12(1):3–23,
1991.
3. W. Cooper, R. Gallupe, S. Pollard, and J. Cadsby. Some
liberating effects of anonymous electronic brainstorming. Small Group Research, 29(2):147, 1998.
16. A. F. Osborn. Applied Imagination, principles and procedures of creative thinking, 1953.
17. A. Pinsonneault, H. Barki, R. B. Gallupe, and N. Hoppen. Electronic brainstorming: The illusion of productivity. Information Systems Research, 10(2):110, 1999.
4. M. Diehl and W. Stroebe. Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of personality and social psychology, 53(3):497,
1987.
18. J. Roland. Questorming: An outline of the method,
1985.
5. P. Dourish and V. Bellotti. Awareness and coordination
in shared workspaces. In Proceedings of the 1992 ACM
conference on Computer-supported cooperative work,
pages 107–114. ACM, 1992.
19. K. Ryall, M. R. Morris, K. Everitt, C. Forlines, and
C. Shen. Experiences with and observations of directtouch tabletops. In Prox. TABLETOP ’06, pages 89–96,
Washington, DC, USA, 2006. IEEE Computer Society.
6. R. Gallupe, L. Bastianutti, and W. Cooper. Unblocking
brainstorms. Journal of Applied Psychology, 76(1):137,
1991.
20. S. D. Scott, M. Sheelagh, T. Carpendale, and
K. M. Inkpen. Territoriality in collaborative tabletop
workspaces. In Proc. CSCW ’04, pages 294–303, New
York, NY, USA, 2004. ACM.
7. F. Guimbretière, M. Stone, and T. Winograd. Fluid interaction with high-resolution wall-size displays. In
Proceedings of the 14th annual ACM symposium on
User interface software and technology, pages 21–30.
ACM, 2001.
21. J. Sosik, B. Avolio, and S. Kahai. Inspiring group creativity. Small Group Research, 29(1):3, 1998.
22. J. Stewart, B. Bederson, and A. Druin. Single display
groupware: a model for co-present collaboration. In
Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, pages
286–293. ACM, 1999.
8. O. Hilliges, L. Terrenghi, S. Boring, D. Kim,
H. Richter, and A. Butz. Designing for collaborative
creative problem solving. In Proceedings of the 6th
ACM SIGCHI conference on Creativity & cognition,
pages 137–146. ACM, 2007.
23. J. Valacich, A. Dennis, and J. Nunamaker. Group size
and anonymity effects on computer-mediated idea generation. Small Group Research, 23(1):49, 1992.
9. U. Hinrichs, M. Hancock, C. Collins, and S. Carpendale. Examination of Text-Entry Methods for Tabletop
Displays. Second Annual IEEE International Workshop
on Horizontal Interactive Human-Computer Systems,
2007.
24. A. VanGundy. Brain Writing for New Product Ideas:
An Alternative to Brainstorming. Journal of Consumer
Marketing, 1(2):67–74, 1993.
25. H. Wang, D. Cosley, and S. Fussell. Idea Expander:
Supporting group brainstorming with conversationally
triggered visual thinking stimuli. In Proc. CSCW ’10,
pages 103–106. ACM, 2010.
10. S. Hunter and P. Maes. WordPlay: A table-top interface for collaborative brainstorming and decision making. Proceedings of IEEE Tabletops and Interactive
Surfaces, 2008.
10
ISBN 978-1-74210-207-8
School of Information Technologies
Faculty of Engineering & Information
Technologies
Level 2, SIT Building, J12
The University of Sydney
NSW 2006 Australia
T +61 2 9351 3423
F +61 2 9351 3838
E [email protected]
sydney.edu.au/it
ABN 15 211 513 464
CRICOS 00026A