Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Participant-driven GSS: Quality of Brainstorming and Allocation of
Participant Resources
Joel H. Helquist
Center for the Management
of Information
University of Arizona
[email protected]
Eric L. Santanen
Department of Management
Bucknell University
[email protected]
John Kruse
Center for the Management
of Information
University of Arizona
[email protected]
Abstract
This paper examines the relationship between time
duration and the quality of brainstorming output.
Quality of brainstorming output is operationalized
using creativity and feasibility measures. Results
indicate that brainstorming quality does decrease
over the duration of the brainstorming session.
Results also indicate the number of off-topic and nonsolution brainstorming output increases significantly
over time. These findings are discussed in light of
participant-driven group support systems.
1. Introduction
Much research has been conducted in the
collaborative domain to further understand the
impacts of GSS on brainstorming and group
productivity. Results have shown that as group size
increases, the benefits of using GSS brainstorming
techniques become more pronounced [1, 2]. Larger
groups using GSS generate more ideas than groups
without GSS support [3, 4]. However, existing
research has not fully addressed the relationship that
exists between the duration of the brainstorming
session and the quality of the brainstorming output.
This paper examines the impact of time on the
productivity of the brainstorming process. The
relationship between brainstorming productivity and
time is one important factor in participant-driven
group support systems (PD-GSS).
2. Participant-driven GSS
Participant-driven group support systems [5]
provide a framework to enable collaboration between
distributed, asynchronous groups. As such, this
framework lays some of the ground work for enabling
large groups to be able to collaborate successfully.
To achieve this end, the PD-GSS framework seeks to
utilize the capacity of the human participants, as well
as systems technology, to reduce the burden on the
facilitator. This framework is related to the research
in collaboration engineering and thinkLets [6, 7]. A
thinkLet constitutes the smallest unit of intellectual
capital required to create one repeatable, predictable
pattern of collaboration among people working
toward a goal [6, 7]. Collaboration engineering
research examines different means to enable the
participants in a collaborative session to execute
collaborative work on their own, mitigating the
dependence on an expert facilitator. The execution of
a successful, asynchronous collaborative session must
utilize similar techniques and methods to empower
the collaborators with the ability to drive the
collaborative activities and reduce the dependence on
a facilitator.
The terminology “participant-driven” refers to the
fact that more of the evaluative and subjective tasks
are completed in parallel by participants of the
distributed, asynchronous collaborative session. This
terminology does not imply that a facilitator is not
necessary or is not utilized during the session. The
role of a session administrator or facilitator to conduct
basic administrative tasks is still necessary for the
successful completion of the collaborative work.
PD-GSS deconstructs the collaborative processes
into discrete segments of work such that members of
the collaborative group are able to receive and process
the work units in an autonomous fashion. The
framework provides an iterative, dynamic process
whereby the system directs the human resources to
the collaborative areas that are the most in need of
work. As the users log into the collaborative session,
the system directs the user to where work is needed,
1530-1605/07 $20.00 © 2007 IEEE
.
1
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
depending on the overall status of the group in the
collaborative process according to the set goals and
parameters that are identified by the facilitator.
One of the critical characteristics of the PD-GSS
framework is that the collaborative activities are
performed in a dynamic, asynchronous workflow.
Traditionally, collaboration workflows utilize a serial
process whereby participants first brainstorm, a
divergent activity, followed by convergent activities,
such as evaluating and ranking the brainstorming
output. PD-GSS creates a workflow that is more
dynamic. Participants may be placed in different
collaborative modules where more resources are
needed. Some participants may be brainstorming
while others are working in other collaborative
modules, perhaps rating or providing feedback on the
brainstorming output. The collaborative modules can
be viewed in terms of thinkLets. Participants are
directed to the various thinkLets as needed, depending
on the status of the group. In this fashion, the
collaborative work is not performed in a serial,
lockstep path. Instead, the workflow is adjusted
according to where the resources are needed.
The dynamic workflow of PD-GSS plays an
important role during divergent activities, as users can
be moved unilaterally to different activities within the
collaborative process. One premise of the dynamic
workflow, with regard to the brainstorming activities,
is that the effectiveness of the divergence phase
declines over time. Effectiveness is defined as the
brainstorming process yielding quality brainstorming
output. It seems reasonable that as the duration of the
brainstorming session increases, the number of quality
brainstorming responses will decline. Users may not
have anything of value to contribute to the
brainstorming pool and either enter “noisy” comments
that are of no value to the group or sit idle at their
workstations. In co-located, synchronous groups, the
facilitator must deal with the “noisy” comments and
the drop in productivity from idle human resources
could be substantial. In an asynchronous, large group,
these kinds of low-quality or no-quality output can
create tremendous overhead for the group due to the
sheer volume of output generated and the time
required to process this additional output. With the
inclusion of this noise, the group is more likely to
become overwhelmed by the sheer number of ideas
generated than when this noise is not present. The
effectiveness and efficiency of the brainstorming
session may be negatively impacted at the end of the
brainstorming session as some of the participants may
still have good ideas to enter while the balance of the
group waits for the next collaborative activity to start.
One of the goals of PD-GSS is to improve
brainstorming productivity by controlling the efficient
allocation of participant resources. As the quality of
brainstorming decreases over time, individual
participants will individually move to other activities
in the collaborative process unilaterally. Eventually,
the quality of the brainstorming output will reach a
threshold at which point the brainstorming module is
closed and all human resources are allocated to
subsequent collaborative activities.
The end result of the PD-GSS brainstorming
design is to have a set of brainstorming output that
has a higher concentration of quality content by
reducing the quantity of “noisy” content [8]. GSS
research has focused much attention on improving the
quantity of brainstorming ideas generated during an
ideation session. However, the link between quantity
and quality of brainstorming is not definite. Research
by Briggs and Reinig [9, 10] has shown that the
causal linkage between quantity and quality indicates
that there are other factors that influence the quality of
the brainstorming output. The PD-GSS framework
posits that as group size increases, achieving a high
number of brainstorming ideas generated is not a
problem. The problem is enabling the collection of
quality brainstorming ideas while limiting the volume
of noise present. Limiting the volume of noise in the
brainstorming pool improves the ability to effectively
manage the information.
This paper examines the relationship between the
quality of the brainstorming output and the duration
of the brainstorming session. This research aims to
provide some validation of the PD-GSS design to
move participants to other activities when
brainstorming productivity declines.
3. Brainstorming quality
This paper utilizes two dimensions to analyze the
quality of the brainstorming output, creativity and
feasibility. These two dimensions have been used to
operationalize quality in previous research on
brainstorming quality [11, 12]. Creativity refers to
the extent to which an idea is out of the ordinary or
novel. Feasibility refers to the extent to which an idea
can be implemented, given the necessary contextual
constraints of the environment.
Additional facets of idea quality have been
proposed and researched [10, 13]. These include such
things as effectiveness and magnitude of impact.
However, for the purposes of this initial exploration
into idea quality over time, we will limit the analysis
to the two facets creativity and feasibility.
3.1 Hypotheses
2
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Exploration of the effects of time passage on
brainstorming quality has received little attention in
the literature. However, it seems reasonable that the
quality of brainstorming will decrease over time for
several reasons. The following section outlines the
literature and the reasons for the decline in
brainstorming quality.
According to the Cognitive Network Model of
Creativity [14], one of the variables affecting the
creation of creative ideas is the cognitive load of the
participants. As the cognitive load of the participants
increases, the ability to be creative and identify
quality solutions decreases as the capacity to process
new ideas is reduced. According to Brigg’s Focus
Theory [9, 15], participants in the collaborative
session have finite attention resources that can be
applied to fulfilling the required task. Thus, as
cognitive load increases, the resources available to be
creative and generate quality brainstorming ideas is
reduced.
During the brainstorming activities, groups
generate large quantities of brainstorming ideas that
must be processed cognitively by the group. As time
passes, this list grows larger and larger. This growing
body of brainstorming ideas can be construed as an
increasing cognitive load on the idea generation
process. While viewing participant’s brainstorming
ideas can lead to novel insight and innovative ideas,
the effort required to read and process all of the
brainstorming ideas may begin to overwhelm
participants, reducing the quality of the ideas.
Likewise, the cognitive effort required increases as
time passes as each participant must process all of the
current ideas in an effort to avoid duplication of
brainstorming ideas.
Participants spend time
analyzing what has been previously submitted,
consuming cognitive resources and potentially
reducing the quality of the ideas that are generated.
We therefore hypothesize the following:
H1: The quality of the solutions generated during
brainstorming activities will decline as a function of
time.
Thus, this research explores this issue with the
intent of guiding further research in this area and
development in GSS that is able to monitor the idea
quality and to shift participants into new activities
when brainstorming quality does not meet a certain
threshold.
Also, in order to investigate the robustness of the
quality performance of the brainstorming groups, two
different tasks were used, each representing different
task types (open versus closed ended problems). This
leads us to the first exploratory research question.
RQ1: Does the type of task (open ended versus
closed ended) impact the quality of brainstorming
output over time?
To further increase the robustness of the findings,
two different brainstorming techniques were
examined (free versus directed brainstorming). This
leads to the second research question.
RQ2: Does the type of brainstorming technique
(free versus directed) impact the relationship between
time and the quality of brainstorming output?
4. Research methods
This section describes the subjects, technology,
dependent and independent variables that were
employed during the present investigation.
4.1 Subjects and technology
Two hundred forty four subjects from upperdivision management information systems courses at
a large public university participated for course credit.
The subjects were randomly arranged into groups
consisting of four individuals each. Each group was
randomly assigned to one of two experimental
conditions (discussed below); each condition used one
of two experimental tasks (also discussed below).
The facilitators read from a prepared script to greet
and introduce the subjects to the computer equipment.
Subjects were subsequently instructed on the use the
electronic brainstorming (EBS) tool of GroupSystems
for Windows, Work Group Edition v2.1 and were
lead through a “warm up” EBS exercise that lasted
approximately five minutes. The EBS software was
configured such that each interactive group of four
subjects had five brainstorming discussion sheets on
which to record their solutions to the problem task.
Setting up one more sheet than there are members in
the group insured that each member would always be
able to exchange their current sheet for a new sheet
after contributing their solution.
4.2 Experimental tasks
Two hypothetical problem tasks were employed
for this research: the Gompin Crisis task and a variant
of the School of Business [16] task. In the Gompin
Crisis task, a small island nation in the middle of the
Pacific Ocean has been devastated by a typhoon. As a
result of the extensive infrastructure damage, 35,000
residents of the Gompin capital are facing a life
3
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
threatening situation because they have been left
without drinkable water for several days. A limited
number of resources are available for disaster relief
efforts; however, there are not enough resources to
supply drinking water to all of the residents of the
capital. The infrastructure damage left in the wake of
the typhoon has placed severe constraints upon the
relief efforts. In this task, the subjects are instructed
to generate solutions that will provide drinkable water
to the residents of the capital within a 48 hour period.
The School of Business task represents a fictitious
situation at a large public university. The subjects are
instructed to generate solutions to a problem
represented by an interconnected set of symptoms,
such as limited physical resources, poor quality
teaching, students who are ill prepared, limited
computer resources, and declining reputation of the
school. The Gompin task is an open-ended problem;
the participants need to think of ways to solve a given
problem that does not have just one predetermined
solution. The School of Business task, however, has a
more narrow solution space than does the Gompin
task, but it is presented in a manner that obscures the
actual source of the symptoms represented in the
problem task.
insure that the groups gave consideration to each of
the relevant sub-dimensions of the problem. Prior
work by Gettys, Pliske, Manning, & Casey [17]
indicate that subjects routinely overlook as much as
80% of the solution space while problem solving.
Further research [18] also indicates that drawing
specific attention to sub-dimensions of the solution
space may be an effective method to increase both the
quantity and quality of solutions that are generated.
Thus, guided by the work of Couger [19], a series of
20 directed brainstorming prompts were derived for
each of the problem tasks (four analogously worded
prompts from each of the following five criteria).
These problem dimensions derive from criteria for
evaluating ideas in a creative problem solving setting
[19]. For the Gompin Crisis task, important solution
criteria are those that:
1. provide sufficient water to the capital
2. can be implemented quickly
3. can be implemented inexpensively
4. are easy to implement
5. use a small quantity of equipment to implement
Correspondingly, important solution criteria for the
School of Business task are those that:
4.3 Brainstorming procedures
In the first experimental treatment (which used the
FreeBrainstorming thinkLet), each member interacted
only with the other members of the group while
brainstorming. Instead of using Osborn’s four rules
for brainstorming (which were intended for verbally
interactive groups), subjects using FreeBrainstorming
followed an alternative set of instructions. The
subjects each submitted one solution to the task at
hand to “seed” the process. The EBS software then
automatically exchanged the participant’s sheet for
another sheet. For each subsequent solution, the
subjects were instructed to read and interact with the
solutions contributed by other members of their group
in one of three ways:
1. Expand on the solution, adding details.
2. Argue with the solution.
3. Contribute a completely new solution.
Groups in the second experimental treatment
(which used the DirectedBrainstorming thinkLet)
used the same EBS software as groups in the
FreeBrainstorming treatment, however, the interaction
procedure varied. Rather than having the subjects
work on their own, a facilitator read a series of
directed brainstorming prompts to the groups every
two minutes. These prompts were intended to help
1. solve the problems faced by the school of
business
2. can be implemented quickly
3. can be implemented inexpensively
4. are easy to implement
5. are acceptable to each of the groups in the
business school
These directed brainstorming prompts were
simultaneously read aloud and displayed on a large
public screen in the front of the room. Subjects were
instructed to listen to the prompt, type in a
corresponding solution, exchange their electronic
sheets, and then read the new solutions appearing on
the newly exchanged sheets. One of the features of
the EBS software allowed each solution to be time
stamped and sequentially numbered, thus facilitating a
detailed analysis of the resulting data. Using two
different
brainstorming
techniques
(free
brainstorming and directed brainstorming) and two
different experimental tasks provided a mechanism to
explore the robustness of the research findings.
4.4 Dependent variables
To examine brainstorming productivity,
following variables were utilized:
the
4
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
• Quantity of off-topic comments: Off-topic
comments are comments that are not relevant to the
problem solving task. Examples from the school of
business task include contributions such as “I don’t
know” and “I am out of ideas.”
• Creativity score: Each solution was individually
scored for creativity (using a five-point Likert scale
where 1=not creative and 5=very creative) by four
experts. In the Gompin task, four disaster relief
specialists from the American Red Cross with an
average of 15.4 years of experience rated the
comments. For the School of Business task, by four
university administrators with an average experience
of 17.6 years rated the comments. Interrater
reliabilities for creativity ranged from 0.834 to 0.910
across tasks and are significant at the 0.001 level.
The individual creativity scores from each of the
experts were then averaged so that each solution had a
corresponding mean creativity score. The following
is one example of a highly creative solution for the
school of business task: “The Business School could
develop and sell software in order to generate money
to contribute towards rising education costs.”
• Technical Feasibility score: Each solution was
individually scored for technical feasibility (using a
five-point Likert scale where 1=not feasible and
5=very feasible) by the experts. Interrater reliabilities
for technical feasibility ranged from 0.805 to 0.904
across tasks and are significant at the 0.001 level.
The individual feasibility scores from each of the
experts were then averaged so that each solution had a
corresponding mean feasibility score. The following
is one example of a highly feasible solution taken
from the school of business task: “Make the results of
the student evaluations count for more by offering
incentives to the professors and TAs who consistently
get high ratings from the students.”
5. Results
Analysis of brainstorming productivity over time
was broken down into three categories: creativity
ratings, feasibility ratings, and the number of off-topic
and non-solution outputs.
below.
Results are presented
5.1 Creativity ratings
A linear regression was performed on the creativity
ratings from the Gompin task (with free brainstorming
and directed brainstorming data combined) to
investigate the relationship between time duration and
creativity.
The results showed that a linear
relationship does exist [F(1,1388)=43.096, p<0.001].
The slope of the regression function is -.031 [t=6.563, p<0.001]. The Y intercept of the regression
function is 2.76 [t=51.290, p<0.001]. The regression
r-squared value was .030. There exists a significant,
negative relationship between time duration and
creativity ratings. As time increases, creativity
declines. However, time duration explains little of the
variance in creativity. See Figure 1.
3.4
Average rating
• Quantity of non-solutions: Non-solutions are on-task
comments that do not directly state a course of action
that is useful in resolving the problem at hand. These
types of output include such things as agreeing with a
solution already entered by another subject or seeking
clarification. Examples from the Gompin data set
include contributions such as “good idea” and
“thanks.”
2.4
1.4
1
3
5
7
9
11
13
15
17
19
Two-minute time segment
Figure 1: Average creativity rating by time
segment - Gompin
Similar analysis was conducted on a divided
Gompin data set to examine the data by treatment. In
the FreeBrainstorming condition, the results showed
that a linear relationship does exist [F(1,177)=7.698,
p<0.05]. The slope of the regression function is -.032
[t=-2.774, p<0.05]. The Y intercept of the regression
function is 2.632 [t=19.359, p<0.001]. The regression
r-squared value was .042.
Data from the
DirectedBrainstorming condition showed similar
results, that a linear relationship does exist
[F(1,1389)=43.069, p<0.001] between creativity and
duration of the brainstorming session. The slope of
the regression function is -.031 [t=-6.563, p<0.001].
The Y intercept of the regression function is 2.758
[t=51.290, p=0.000]. The regression r-squared value
was .030.
For the School of Business task, the same
regressions were performed. The results showed that
a linear relationship does exist [F(1,1343)=44.529,
p<0.001] between creativity and duration of the
brainstorming session. The slope of the regression
5
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
3.4
Average Rating
explains very little of the variance in creativity. See
Figure 3.
3.4
Average rating
function is -.030 [t=-6.673, p<0.001]. The Y intercept
of the regression function is 2.740 [t=51.709,
p=0.000]. The regression r-squared value was .032.
Like the Gompin data, there exists a negative
relationship between time duration and creativity
ratings. See Figure 2.
2.4
2.4
1.4
1
3
5
7
9
11
13
15
17
19
Two-minute time segment
1
3
5
7
9
11
13
15
17
19
Two-minute time segment
Figure 2: Average creativity rating by time
segment - School of Business
Analysis by treatment was conducted on the
School of Business data. In the FreeBrainstorming
condition, no significant linear relationship exists
between
creativity
and
passage of
time
[F(1,187)=0.003,
p=0.957].
In
the
DirectedBrainstorming condition, a linear relationship
exists [F(1,1155)=49.412, p<0.001]. The slope of the
regression function is -.0346 [t=-7.029, p<0.001].
The Y intercept of the regression function is 2.804
[t=47.619, p<0.001].
We can see from these regression analyses that the
overall creativity ratings decline over time, with the
exception of the FreeBrainstorming condition for the
School of Business Task. Specifically, in the Gompin
task, creativity ratings decline by 22% over the 40
minute brainstorming period. In the school of
business scenario, the creativity ratings also decline
by 22% over the duration of the brainstorming
session.
5.2 Feasibility ratings
Linear regression was performed on the Gompin
feasibility scores to examine the relationship between
time duration and feasibility scores. The results
showed that a linear relationship does exist
[F(1,1388)=94.587, p<0.001]. The slope of the
regression function is -.041 [t=-9.726, p<0.001]. The
Y intercept of the regression function is 2.52
[t=51.841, p<0.001]. The regression r-squared value
was .064.
There exists a significant, negative
relationship between time duration and feasibility
ratings. As time increases, feasibility scores decline.
However, as with creativity ratings, time duration
Figure 3: Average feasibility ratings by twominute time interval - Gompin
Analysis of each treatment was performed using
the same procedures. In the FreeBrainstorming
condition,
a
linear
relationship
exists
[F(1,177)=19.383, p<0.001].
The slope of the
regression function is -.053 [t=-4.403, p<0.001]. The
Y intercept of the regression function is 2.963
[t=30.729, p<0.001]. The regression r-squared value
was .099. In the DirectedBrainstorming condition,
the results showed similar results, that a linear
relationship does exist [F(1,1389)=94.587, p<0.001].
The slope of the regression function is -.041 [t=9.726, p<0.001]. The Y intercept of the regression
function is 2.525 [t=51.841, p=0.000]. The regression
r-squared value was .064.
For the School of Business task, the same
regression on feasibility was performed. The results
showed that a linear relationship does exist when the
data are combined [F(1,1343)=71.902, p<0.001]. The
slope of the regression function is -.035 [t=-8.480,
p<0.001]. The Y intercept of the regression function
is 2.40 [t=49.633, p<0.001]. The regression r-squared
value was .051. Like the Gompin data, there exists a
negative relationship between time duration and
feasibility ratings. See Figure 4.
3.4
Average Rating
1.4
2.4
1.4
1
3
5
7
9
11
13
15
17
19
Two-minute time segment
6
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
The analysis was repeated by treatment for the
School of Business task. In the FreeBrainstorming
condition, no linear relationships was found
[F(1,187)=1.955, p=0.164].
In the treatment
condition, a linear relationship was found between
feasibility and the passage of time [F(1,1155)=84.665,
p<0.001]. The slope of the regression function is .041 [t=-9.201, p<0.001]. The Y intercept of the
regression function is 2.45 [t=46.014, p<0.001]. The
regression r-squared value was .068.
The regression analyses indicate that the overall
feasibility ratings decline over time, again, with the
exception of the FreeBrainstorming condition for the
School of business task. Specifically, in the Gompin
task, feasibility ratings decline by 33% over the 40
minute brainstorming period. In the school of
business scenario, the feasibility ratings decline by
29% over the duration of the brainstorming session.
5.3 Off-topic and non-solutions
To analyze the impact of the passage of time on
the quantity of off-topic comments and non-solutions,
a regression analysis was performed on the combined
data (FreeBrainstorming and DirectedBrainstorming)
for each task. For the Gompin task, the results
showed that a linear relationship does exist between
time duration and the quantity of off-topic and nonsolutions [F(1,18)=70.169, p<0.001]. The slope of
the regression function is 0.401 [t=8.389, p<0.001].
The Y intercept of the regression function is .791
[t=1.381, p=0.184]. The regression explains 79.6
percent of the variance [r-squared = 0.796] in the
relationship between the frequency of non-solution
and off topic comments and time duration. Each
successive two-minute interval increased the
frequency of off-topic and non-solutions by
approximately 0.4 percent (see Figure 5).
The data was analyzed by treatment in the same
manner. In the FreeBrainstorming condition, the
same
linear
relationship
was
observed
[F(1,18)=25.977, p<0.001].
The slope of the
regression function is 0.695 [t=5.097, p<0.001]. The
Y intercept of the regression function is 5.847
[t=3.577, p<0.05]. The regression explains 59.1
percent of the variance [r-squared = 0.591]. In the
DirectedBrainstorming condition, the same results
were observed. A linear relationship existed between
time and the number of off-topic and non-solutions
[F(1,18)=70.043, p<0.001].
The slope of the
regression function is 2.032 [t=8.369, p<0.001]. The
Y intercept of the regression function is 3.810
[t=1.310, p=0.207]. The regression explains 79.6
percent of the variance [r-squared = 0.796].
Regression analysis was also performed on the
combined data from the School of Business task. A
linear relationship exists between time and the
frequency
of
off-topic
and
non-solutions
[F(1,16)=71.916, p<0.001]. The slope is 0.715
[t=8.480, p<0.001]. The Y intercept of the regression
function is -2.50 [t=-2.536, p=0.032]. This regression
function explains 81.8 percent of the variance [rsquared = 0.818] in the relationship between the
frequency of non-solution and off topic comments and
time duration. Like the Gompin scenario, each
successive time interval increased the frequency of off
topic and non-solutions by approximately 0.7 percent
(see Figure 6).
14.0
12.0
10.0
Percent
Figure 4: Average feasibility ratings by twominute time interval - School of Business
8.0
6.0
4.0
2.0
0.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Two-minute time segment
12
Figure 6: Percentage of off-topic and nonsolutions for the School of Business task
10
Percent
8
6
4
2
0
1
3
5
7
9
11
13
15
17
19
Two-minute time segment
Figure 5: Percentage of off-topic and nonsolutions for the Gompin task
The regression analysis was also conducted on the
School of Business task by treatment. The results are
consistent with the previous findings.
In the
FreeBrainstorming condition, a linear relationship
exists F(1,18)=14.588, p<0.001]. The slope of the
regression function is 0.379 [t=3.819, p<0.05]. The Y
intercept of the regression function is 5.22 [t=4.287,
p<0.001]. The regression explains 46.2 percent of the
variance [r-squared = 0.462].
In the
7
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
DirectedBrainstorming condition, similar results were
observed. A linear relationship existed between time
and the number of off-topic and non-solutions
[F(1,18)=101.391, p<0.001].
The slope of the
regression function is 1.784 [t=10.069, p<0.001]. The
Y intercept of the regression function is -6.218 [t=2.858, p<0.001].
The regression explains 85.6
percent of the variance [r-squared = 0.856].
Overall, in the Gompin task, the percentage of offtopic and non-solutions increases 465% from the
beginning to the end of the task. In the school of
business task, the percentage of off-topic and nonsolutions increases 573% over the 40 minute time
interval.
6. Discussion
Hypothesis 1 predicted that the quality of the
brainstorming ideas generated by the group declines
as a function of time. Significant linear relationships
were identified in both the Gompin and the School of
Business tasks, indicating a negative relationship
between brainstorming quality and time. Overall,
both experimental tasks showed evidence of a
decrease in brainstorming quality over the duration of
the task, lending support for the hypothesis.
Research question 1 examined the impact of the
type of task (closed versus open-ended) on the decline
in brainstorming quality. The overall analysis reveals
that both closed and open-ended tasks yield similar
results. In our experiment, both tasks exhibited
declining quality as a function of time.
Research question 2 examined the impact of free
brainstorming versus directed brainstorming to see if
both treatments exhibited the same characteristics
with regards to brainstorming quality over time. The
data from the two tasks provides support for the idea
that both forms of brainstorming are susceptible to
declining brainstorming quality over time.
The exception to these results is that both
creativity and feasibility results from the School of
Business task show no linear relationship at all with
the passage of time in the FreeBrainstorming session.
This finding suggests that perhaps some characteristic
of either the School of Business task itself (a closedended task) or the FreeBrainstorming methodology is
responsible for this outcome.
Clearly, further
research is needed to help discover the cause of this
interesting outcome.
These results provide some empirical support for
the notion that brainstorming quality tends to decline
as a function of time and that this phenomenon is
fairly robust. The effects hold across different
contextual tasks as well as types of brainstorming
methods.
These results support the notion of a
Participant-driven GSS workflow, whereby the
participants are able to work dynamically, in different
collaborative modules.
As the quality of the
brainstorming ideas declines, users are able to
contribute in other ways. In this manner, the volume
of off-topic and non-solutions is reduced and the
group is not forced to deal with the overhead that they
incur.
7. Conclusion
The results from this paper provide support for the
premise that the creativity and feasibility of
brainstorming input decreases over time while the
number of off-topic and non-solutions increases.
These results highlight some important ideas for both
facilitators as well as GSS designers. First, the results
underscore the importance of the facilitator or
meeting leader selecting an appropriate length of time
for brainstorming. Depending on the context of the
meeting, proximal versus distributed and synchronous
versus asynchronous, the volume of noisy input may
be a serious detriment to the group’s productivity.
For the GSS designers, the decrease in the quality
of brainstorming output illustrates the need for a
mechanism in the collaborative software to monitor
the brainstorming process and the ratings of the ideas
generated. This monitoring could be a system-based
mechanism that possesses a threshold to close down
brainstorming activities after a certain level of quality
is breached. Likewise, the facilitator of the group
could monitor the brainstorming ratings of the group
and be able to manually control the number of
participants that are able to still brainstorm, moving
other participants to different activities where their
resources can be utilized more effectively.
Significant research needs to be conducted to further
explore the implementation of these options in a GSS
as well as understanding their impacts on the
collaborative process and outcome.
Shifting users to subsequent activities when the
threshold is crossed could be accomplished in two
ways. First, once a certain level is reached, no
participants are allowed to enter any new
brainstorming ideas. The users’ screens would
automatically be transitioned to a different
collaborative activity. Alternatively, the number of
participants could be gradually phased out. In this
scenario, participants may be given the option to enter
a new brainstorming idea if they feel they have
something of value to contribute. If the participant
does not have any new brainstorming ideas to enter,
the system will direct the participant to another
activity. One interesting prospect associated with this
8
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
is to perform ongoing analysis of user contributions
throughout the brainstorming session. The system
could analyze the participants to identify those that
have provided or are providing brainstorming output
that is rated highly and those that are not. The system
could be set to automatically identify and move these
less productive brainstorming participants to other
activities where they could be more productive. More
research is needed to further examine these potential
scenarios.
8. References
[1] Fjermestad, J. and Hiltz, S. R., "An assessment of
group support systems experiment research: Methodology
and results," Journal of Management Information Systems,
vol. 15, 1998, 7.
[2] Gallupe, R. B., Dennis, A. R., Cooper, W. H.,
Valacich, J. S., Bastianutti, L. M., and Nunamaker, J. F., Jr.,
"Electronic Brainstorming and Group Size," Academy of
Management Journal, vol. 35, 1992, 350-369.
[3] Dennis, A. R. and Valacich, J. S., "Computer
brainstorms : More heads are better than one," Journal of
Applied Psychology, vol. 78, 1993, 531-537.
[4] Nunamaker, J. F., Alan, R. D., Joseph, S. V., Douglas,
V., and Joey, F. G., "Electronic meeting systems to support
group work," Communications of the ACM, vol. 34, 1991,
40-61.
[5] Helquist, J. H., Kruse, J., and Adkins, M.,
"Developing large scale participant-driven group support
systems: An approach to facilitating large groups.,"
presented at Proceedings of the First HICSS Symposium on
Field and Case Studies of Collaboration, Los Alamitos, CA,
2006.
quantity in electronic brainstorming," presented at
Proceedings of the Thirtieth Hawaii International
Conference on Systems Sciences, 1997.
[10] Reinig, B. A. and Briggs, R. O., "Measuring the
Quality of Ideation Technology and Techniques," presented
at Proceedings of the 39th Annual Hawaii International
Conference on System Sciences, 2006.
[11] Diehl, M. and Stroebe, W., "Productivity loss In
brainstorming groups : Toward the solution of a riddle,"
Journal of Personality and Social Psychology, vol. 53,
1987, 497-509.
[12] Diehl, M. and Stroebe, W., "Productivity loss in ideagenerating groups : Tracking down the blocking effect,"
Journal of Personality and Social Psychology, vol. 61,
1991, 392-403.
[13] Barki, H. and Pinsonneault, A., "Small group
brainstorming and idea quality: Is electronic brainstorming
the most effective approach?," Small Group Research, vol.
32, 2001, 158-205.
[14] Santanen, E. L., Briggs, R. O., and Vreede, G. J., de,
"Causal relationships in creative problem solving:
Comparing facilitation interventions for ideation," Journal
of Management Information Systems, vol. 20, 2004, 167198.
[15] Briggs, R. O., "The focus theory of group productivity
and its application to development and testing of electronic
group support systems," Ph.D. dissertation, University of
Arizona, Tucson, 1994.
[16] Mennecke, B. E. and Wheeler, B. C., "Tasks matter:
modeling group task processes in experimental CSCW
research," Proceeding of the Twenty-Sixth Hawaii
International Conference on System Sciences, 1993.
[6] Briggs, R. O., Vreede, G. J., de, and Nunamaker, J. F.,
Jr., "Collaboration engineering with thinkLets to pursue
sustained success with group support systems," Journal of
Management Information Systems, vol. 19, 2003, 31-64.
[17] Gettys, C. F., Pliske, R. M., Manning, C., and Casety,
J. T., "An evaluation of human act generation performance,"
Organizational Behavior and Human Decision, vol. 39,
1987, 23-51.
[7] de Vreede, G. J. and Briggs, R. O., "Collaboration
Engineering: Designing Repeatable Processes for HighValue Collaborative Tasks," presented at Proceedings of the
38th Annual Hawaii International Conference on System
Sciences, 2005.
[18] Dennis, A. R., Valacich, J. S., Carte, T. A., Garfield,
M. J., Haley, B. J., and Aronson, J. E., "Research report:
The effectiveness of multiple dialogues in electronic
brainstorming," Information Systems Research, vol. 8, 1997,
203-211.
[8] Helquist, J. H., Kruse, J., and Adkins, M., "Group
support systems for very large groups: A peer review
process to filter brainstorming input," in Proceedings of the
Americas Conference on Information Systems. Acapulco,
Mexico, 2006.
[19] Couger, J. D., Creative Problem Solving and
Opportunity Finding. Danvers, MA: Body & Fraser
Publishing Company, 1995.
[9] Briggs, R. O., Reinig, B. A., Shepherd, M. M., Yen,
J., and Nunamaker, J. F., Jr., "Quality as a function of
9