Stopping Rule Use During
Information Search in
Design Tasks
Glenn J. Browne
Texas Tech University
Mitzi G. Pitts
University of Memphis
Outline

Introduction: Aims of the Study
 Background
 Setting and Method
 Results
 Conclusion
Introduction: Aims of the Study

Improving requirements determination for
systems development
 Understanding how analysts decide when to
stop gathering information
 Understanding role of experience in that
decision
Background:
The Decision-Making Process

Simon’s Model
– Intelligence
– Design
– Choice
Information Search

Why we might expect information search in
different stages of the decision-making process to
differ.

“Stopping Rules”
Information Acquisition

Problems in acquisition
– Underacquiring
– Overacquiring
Heuristics
Defined – rules of thumb for taking actions
in various situations
 Examples

– “80-20 Rule”
– “Feed a cold, starve a fever”
Heuristics for Assessing
Likelihood
Normative – Relative Frequency
 Descriptive

– E.g., availability, representativeness, anchoring
and adjustment
Heuristics for Choice

Normative
– E.g., expected value of information, expected
value of additional information, expected loss
from terminating information acquisition.

Descriptive
– E.g., Dominance, Conjunctive, Disjunctive,
“The Minimalist,” “Take the Best.”
Heuristics for Intelligence
Gathering and Design
?
Heuristics for Intelligence Gathering
and Design:
Some Ideas

Nickles, Curley, and Benson (1995)
– Difference Threshold
– Magnitude Threshold
– Mental List
– Representational Stability
Difference Threshold Stopping Rule
Magnitude Threshold Stopping Rule
Mental List Stopping Rule
Representational Stability Stopping Rule
Impact of Analyst Experience

On information gathered
 On stopping rules used
The Context

Requirements gathering for information systems
development
 Application for grocery shopping on world wide
web
 54 practicing systems analysts in the BaltimoreWashington metro area
 Participants asked to gather requirements until
they felt they had enough information to draw
diagrams representing requirements and proceed
with system design.
Measuring Information
Requirements

Requirements Taxonomy
 Total requirements (Quantity)
 Breadth
 Depth
Hypotheses



H1a: The use of some stopping rules will result in
different quantities of requirements than the use of
others.
H1b: The use of some stopping rules will result in
different breadth of requirements than the use of
others.
H1c: The use of some stopping rules will result in
different depth of requirements than the use of
others.
Hypotheses (cont.)




H2a: A greater number of experienced analysts will
use the mental list rule than will use the
representational stability rule.
H2b: A greater number of experienced analysts will
use the mental list rule than will use the difference
threshold rule.
H2c: A greater number of experienced analysts will
use the magnitude threshold rule than will use the
representational stability rule.
H2d: A greater number of experienced analysts will
use the magnitude threshold rule than will use the
difference threshold rule.
Hypotheses (cont.)


H3a: There will be no relationship between the
experience of the analyst and the quantity of
requirements elicited.
H3b: There will be no relationship between the
experience of the analyst and the quality of
requirements elicited.
Data Analysis

Verbal protocols and questionnaires
 Coding
 Interrater reliability
 Stopping rule identification
Results

Stopping Rule Use
– Difference Threshold – 22
– Representational Stability – 13
– Mental List - 10
– Magnitude Threshold – 9
Results (cont.)

Requirements Elicited by Stopping Rule
– Quantity – F(3,50) = 2.72; p = .05
– Breadth - F(3,50) = 1.72; p = .17
– Depth - P2(3) = 8.98; p = .03
Results (cont.)

Impact of Experience on Stopping Rule Use
– Mental List = 14.30 years
– Magnitude Threshold = 14.06 years
– Difference Threshold = 11.11 years
– Representational Stability = 7.65 years
Results (cont.)

Impact of Experience on Stopping Rule Use
– Mental List rule users were more experienced than
users of the Representational Stability rule (t(21) = 2.27;
p = .019), supporting Hypothesis 2a.
– Users of the Magnitude Threshold rule were also more
experienced than users of the Representational Stability
rule (t(20) = 2.00; p = .03), supporting Hypothesis 2c.
– Other two hypotheses were not supported.
Results (cont.)

Impact of Experience on Requirements
Elicited
– Analysts’ years of experience were unrelated to
the total number of requirements elicited
(Pearson’s r2 = .08; p = .59), supporting H3a.
– Breadth of requirements (r2 = .15; p = .27) and
depth of requirements (r2 = .02; p = .91) were
also unrelated to analysts’ years of experience,
supporting H3b.
Conclusion

Identification of stopping rules during
information search
 Impacts of analyst experience
 Impact on information systems
development process