Towards a Model for Creative
Requirements Prioritization. A Design
Research in Requirements
Prioritization for the Gaming Industry
A. S. Chervenkov
August 2013
Keywords: requirements prioritization, software product management, gaming industry, creative element of video game development, creativity, preservation of creative vision, agile development
Abstract
Using gathered knowledge from academic literature research in the
field of creativity and the field of software product management and
combining it with empirical research in the form of conversation and
collaboration with experts from the gaming industry, we were able to
construct a list of design criteria that served as the blueprint for the
development of the Creative Requirements Prioritization. The CRP
is a two-component technique consisting of the CRP Model responsible for the definition and preservation of a shared core creative vision throughout the development process and the CRP method for
intelligent ordering of requirements based on evaluating requirement’s
importance to the core creative vision.
Contents
1 Introduction
1
1.1
Research trigger . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.2
Research scope and goal . . . . . . . . . . . . . . . . . . . . .
2
1.3
Research question and sub-questions . . . . . . . . . . . . . .
3
1.4
Scientific relevance . . . . . . . . . . . . . . . . . . . . . . . .
4
1.5
Practical relevance . . . . . . . . . . . . . . . . . . . . . . . .
5
2 Research approach
6
2.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
2.2
Research stages . . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.2.1
Stage I: Literature review . . . . . . . . . . . . . . . .
8
2.2.2
Stage II: Expert’s point of view . . . . . . . . . . . . .
9
2.2.3
Stage III: Develop a CRP solution . . . . . . . . . . .
9
2.2.4
Stage IV: Evaluation . . . . . . . . . . . . . . . . . . .
10
2.2.5
Stage V: Documentation . . . . . . . . . . . . . . . . .
10
Formalized overview . . . . . . . . . . . . . . . . . . . . . . .
10
2.3
3 Inquiries rationale and strategy
3.1
3.2
14
Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
3.1.1
Design strategy . . . . . . . . . . . . . . . . . . . . . .
14
3.1.2
Data collection and fieldwork strategy . . . . . . . . .
15
3.1.3
The pitfalls of qualitative/naturalistic inquiry . . . . .
16
The questioning . . . . . . . . . . . . . . . . . . . . . . . . . .
17
3.2.1
Questionnaire, interview or both? . . . . . . . . . . . .
17
3.2.2
Closed vs. open questions . . . . . . . . . . . . . . . .
18
Closed questions advantages . . . . . . . . . . . . . . .
18
Closed questions disadvantages . . . . . . . . . . . . .
18
Open questions advantages . . . . . . . . . . . . . . .
19
Open questions disadvantages . . . . . . . . . . . . . .
19
i
4 Theoretical foundations
4.1
21
The creative prespective . . . . . . . . . . . . . . . . . . . . .
22
4.1.1
Definition of creativity . . . . . . . . . . . . . . . . . .
22
4.1.2
Research on creativity . . . . . . . . . . . . . . . . . .
23
4.1.3
Csikszentmihalyi’s Systems Model of Creativity . . . .
24
4.1.4
Creative element in entertainment software . . . . . .
28
The technical prespective . . . . . . . . . . . . . . . . . . . .
29
4.2.1
A brief history of video games . . . . . . . . . . . . . .
30
4.2.2
Overview of video game development . . . . . . . . . .
32
4.2.3
Agile development: Scrum . . . . . . . . . . . . . . . .
36
Sprint overview . . . . . . . . . . . . . . . . . . . . . .
39
Overview of typical game development complications .
39
Requirements prioritization . . . . . . . . . . . . . . . . . . .
42
4.3.1
Requirements Triage . . . . . . . . . . . . . . . . . . .
44
4.3.2
Analytical Hierarchy Process (AHP) . . . . . . . . . .
45
4.3.3
Cumulative Voting . . . . . . . . . . . . . . . . . . . .
47
4.3.4
Numerical Assignment . . . . . . . . . . . . . . . . . .
48
4.3.5
Ranking . . . . . . . . . . . . . . . . . . . . . . . . . .
48
4.3.6
Top 10 Requirements . . . . . . . . . . . . . . . . . . .
49
4.4
Creative element support in requirements prioritization . . . .
49
4.5
Creative element effects on requirements prioritization . . . .
52
4.2
4.2.4
4.3
5 Empirical findings
5.1
5.2
54
Approaching the data . . . . . . . . . . . . . . . . . . . . . .
54
5.1.1
From questionnaire to conversation . . . . . . . . . . .
56
Key expert notions and attitudes . . . . . . . . . . . . . . . .
59
5.2.1
Creativity and "enjoyment" are fundamentally not quantifiable . . . . . . . . . . . . . . . . . . . . . . . . . . .
60
5.2.2
Process implies rigidity . . . . . . . . . . . . . . . . . .
60
5.2.3
Process kills creativity . . . . . . . . . . . . . . . . . .
61
5.2.4
Process is just a shield that shifts responsibility . . . .
62
5.2.5
Profitability vs. risk is the only important metric . . .
62
ii
5.2.6
Creative vision works only for art . . . . . . . . . . . .
63
5.2.7
Creativity is a group activity . . . . . . . . . . . . . .
64
5.2.8
The gaming industry is a hot bed of imitation . . . . .
65
5.2.9
Introduction of novel ideas requires "champions" . . .
66
5.2.10 Feature creep is the result of the absence of a shared
5.3
vision . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
5.2.11 Organization culture determines creativity flow . . . .
68
Industry expert guidelines for CRP . . . . . . . . . . . . . . .
69
6 Creative Requirements Prioritization
71
6.1
Design criteria and rationale . . . . . . . . . . . . . . . . . . .
71
6.2
CRP Model . . . . . . . . . . . . . . . . . . . . . . . . . . . .
74
6.2.1
Example of CRP in use under Scrum . . . . . . . . . .
77
6.2.2
Discussion - satisfying the design criteria . . . . . . . .
82
Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
85
6.3
7 Conclusion, limitations and future work
87
References
90
iii
List of Figures
1
Research scope – A layered representation illustrating the knowledge flow from one concept to another until reaching the main
research goal. . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
Reasoning in the Design Research Cycle of Vaishnavi & Kuechler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
3
7
Detailed research approach illustrated using a Process-Deliverable
Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
4
Outermost layer of the research scope diagram: define creativity. 21
5
Mihaly Csikszentmihalyi’s Systems Model of Creativity [21] .
6
Outermost layer of the research scope diagram: creativity in
enterainment software. . . . . . . . . . . . . . . . . . . . . . .
7
26
Inner layer of the research scope diagram: creative element in
enterainment software. . . . . . . . . . . . . . . . . . . . . . .
8
25
29
Tennis for Two - (arguably) the second video game ever. Created by Willy Higinbotham in 1958 at Brookhaven National
Laboratory . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
9
Refrence Method for Game Production: Process Overview [79]
35
10
Scrum - agile software develepment . . . . . . . . . . . . . . .
37
11
Outermost layer of the research scope diagram: requirements
prioritization techniques. . . . . . . . . . . . . . . . . . . . . .
12
42
Triage - A primitive method of quickly ordering patients by
severity of their heath condition. Picture: Triage station,
Suippes, France, World War I. . . . . . . . . . . . . . . . . . .
13
Outermost layer of the research scope diagram: creative element support in requirements prioritization. . . . . . . . . . .
14
44
50
Inner layer of the research scope diagram: creative element
effects on requirements prioritization. . . . . . . . . . . . . . .
52
15
The Data Analysis Process according to Seidel [71] . . . . . .
56
16
Design criteria rationale of the Creative Requirements Prioritization techniqe . . . . . . . . . . . . . . . . . . . . . . . . .
17
72
Core of the research scope diagram: final step (3) - desing a
Creative Requirements Prioritization solution . . . . . . . . .
iv
74
18
Creative Requirements Prioritization Model . . . . . . . . . .
19
Prioritizing using the Creative Requirements Prioritization Model 77
20
CRP Model building phase illustrated using a Process-Deliverable
Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
v
75
78
List of Tables
1
Design Science Research in Information Systems guidelines as
proposed by Hevner et. al. . . . . . . . . . . . . . . . . . . . .
8
2
Research approach activities as seen in the PDD (see Figure 3). 12
3
Research approach concepts as seen in the PDD (see Figure 3). 13
4
An example of Analytical Hierarchy Process (AHP) in use. . .
5
Comparison of popular requirements prioritization techniques
46
based on measurement scale, granularity of analysis, and level
of sophistication of the technique [12] . . . . . . . . . . . . . .
51
6
Table of respondents . . . . . . . . . . . . . . . . . . . . . . .
59
7
CRP’s activities table as seen in the PDD (see Figure 20). . .
79
8
CRP’s concepts table as seen in the PDD (see Figure 20). . .
79
9
Ordering requirements using the CRM Model
. . . . . . . . .
81
vi
1
Introduction
This design research conducted at the Utrecht University is the main part
of a Business Informatics master thesis project. The research explores and
shines a light on the common practices of today’s video game development
processes, in particular the domain of requirements management.
The research problem investigated is the seeming absence of literature on
specific requirements management solutions for the gaming industry. Paradoxically, regular software product management (SPM) offers a plethora of
mechanisms aiming at streamlining and enhancing the software development
process, but few can be applied to video game development. The intent is
to study popular SPM literature and interact with video-game developers in
order to identify the common ground between literature and actual practices.
Ultimately, using this knowledge, a method for requirements prioritization
suited to the specific necessities of video game production process is produced.
1.1
Research trigger
The broad research field of software product management has supplied software product managers with various ways of streamlining most development
processes and different performance benchmarking techniques. Theoretically, the scope of the SPM Competence model [78] encompasses the whole
life cycle of any software product, from its inception to customer delivery in
order to generate the biggest possible value to the business. However, one of
the most profitable software industries, the video-game industry [7] is also
one of the most overlooked when it comes to SPM research and especially
requirements gathering and prioritization techniques.
Game development is in many ways similar to product software development
as developers follow certain software development processes. Following a
poor development method (or none at all) could result in longer development
times, going over budget and/or delivering buggy products [13]. What makes
video games different is the creative game vision that must be shared by the
entire team to ensure that the end product is consistent and of good quality.
This is especially true for full-blown game titles where from a user point of
view, there are more similarities with film art than with any other software.
1
Unfortunately, this curious creative aspect renders many SPM techniques
unadoptable by the gaming industry.
This thesis project aims at exploring the differences between regular software
and entertainment software development. More specifically, the focus is on
defining and understanding the creative aspects of requirements during the
initial conceptualization and planning stage of game development.
1.2
Research scope and goal
The goal of this research has three main emphases: (1) understanding the
creative element of entertainment software; (2) realize how this creative element of entertainment software affects development in regards to requirements prioritization; (3) utilize the knowledge in developing a requirements
prioritization solution facilitating the specificities (i.e. the creative element)
of entertainment software.
Figure 1 illustrates the scope of the research at hand in a layered fashion,
demonstrating the knowledge flow between the related concepts, regardless
of the source of information. The idea is to carry out the research in a
systematic way by starting with the outermost layer and slowly “peeling”
until the core research concept is reached. Following this system should
allow for the thorough investigation expected when dealing with exploratory
research.
2
5*"+&$,
$&-($
%("
%&
"*
;>=
%
"
&
(
"
%
$
*&+$%
"%&(
7"
"7
6
%&
"
(
;<=
%(. /
0
5*"+&$,"
*"23$*"7"%&4
8*$'*$&$9+&$'%
&"
% $2
('%
;?=
3"
4
5*"+&
(
8'
6" 7 "%&("##")&4
38
"("
&(4
0(
)1
&$,
./
5*
./
"+
* &($
0
5*"
+&$
,"
!"#
$%$
&$'
%(
+*"
#&:
(4'
#&:+*"
&(4'
"%
$%7
&+
$&+&$,
"
*
()
'#
$, "
"7
( "6
"%
0(*"23$*"7"%&4
((8*$'*$&$9+&$'%
Figure 1: Research scope – A layered representation illustrating the knowledge
flow from one concept to another until reaching the main research goal.
Our research studies a fairly unexplored area of software product management and in order to stay focused and deliver, a well-defined research question is due.
1.3
Research question and sub-questions
The main research question of the research at hand, synthesizes the concepts
expressed in Figure 1:
RQ: How can a requirements prioritization technique facilitate
and nurture the creative aspects of video game software
development?
Obviously this question is too complex to be answered without decomposing
it first. Here is a list of research sub-questions that need to be answered in
order to explore the universe of the main research question:
3
? SQ1: What is creativity?
– Creativity is a very broad subject that is not in any way exclusive
to the field of Information Science. Nevertheless, it is crucial to
clearly define the meaning of creativity and formulate it, as it is
a pivotal element in the research at hand. The answer to this
sub-question is part of chapter 4.1.
? SQ2: How does creativity relate to video game software?
– With the characteristics of creativity outlined, the next step is
to see why exactly are video games different from spreadsheet
software. Of course, this has to be done keeping in mind the
relation with requirements engineering. The answer to this subquestion is part of chapter 4.1.
? SQ3: Why are established requirements prioritization techniques inadequate for video game development?
– It is important to identify the established requirements prioritization techniques and their characteristics, as it is an integral part
of the main research question. Answering this question requires
studying available literature on the matter as well as tapping into
real world experiences. The answer to this sub-question is part of
chapter 4.2
In the process of answering the sub-questions above, we can gather the requirements needed for the creation of a domain-specific requirements prioritization solution tailored to the specific needs of game development. The
open nature of the questions in this research is aimed at facilitating the initial exploratory search in a field that has seen little or none attention by the
academia. The limitations of this research are discussed at greater length
later on.
1.4
Scientific relevance
There are several scientific contributions this research aims at delivering. As
previously mentioned, the literature on game development currently has very
4
little to say about the pre-production stage of requirements management and
especially requirements prioritization. This is not to say there is no need,
on the contrary. Problems easily associated with inadequate requirements
prioritization such as scheduling problems, quality problems, scope problems
etc. are among the most common reasons for unsuccessful software projects
[8, 13, 25, 26]. Shining light on requirements prioritization in the gaming
industry by summarizing and discussing the available academic knowledge in
the domain on one hand, and investigating real world practices on the other,
should help tackle all of those major problem areas in the development process by providing a base for further research. Moreover, this research aims
at producing a requirements prioritization solution, incorporating the knowledge acquired from investigating the real-world practices combined with the
available knowledge from the software product management literature.
1.5
Practical relevance
The practical contribution is quite straightforward – a situational tool designed specifically for the needs of the gaming industry - a multibillion-dollar
industry [7]. Taking into account what sets video games apart from conventional computer software (i.e. the creative aspect), the requirements prioritization tool developed should help minimize the chance of dealing with what
are considered the most costly problems in the industry. A need for easy
to apply, robust method for defining the core requirements of video game
projects is apparent. By applying the principles of research design, the solutions proposed in this work are jointly developed with the help of active
experts in the industry.
5
2
Research approach
This section of the paper discusses the mechanics of the research in greater
detail, showing the steps taken and the deliverables that will be produced
in the process of developing a Creative Requirements Prioritization (CRP)
solution.
2.1
Overview
Two paradigms characterize much of the research in the Information Systems discipline: behavioral science and design science. Behavioral-science
paradigm seeks to develop and verify theories that explain or predict human
or organizational behavior whereas design-science paradigm seeks to extend
the boundaries of human and organizational capabilities by creating new and
innovative artifacts [32]. Since the key product of this research is the development of a requirements prioritization solution (the artifact), it is classified
as a design research. The common understanding of Design Science Research
in Information Systems (DSRIS) continues to evolve, and the only consensus
is that DSRIS is a way of learning through the act of building [77]. Figure 2
basicly provides an overview of the design cycle followed in this research as
proposed by Vaishnavi and Kuechler [77].
6
Circumscription
Operation and goal Knowledge
Knowledge Flows
Process Steps
Outputs
Awareness of Problem
Research question
Suggestion
Literature research
Expert’s POV
Development
Develop CRP solution
Evaluation
Validate CRP solution
Conclusion
Documentation
Template(s)
Paper
Figure 2: Reasoning in the Design Research Cycle of Vaishnavi & Kuechler
In the design research cycle (see Figure 2 ), the research is presented in
steps (Process steps) with corresponding outputs. Some of the steps produce
new knowledge that is communicated back (illustrated by the knowledge
flow), which could results in altering hypotheses or - in extreme cases - even
abolishing the research altogether.
2.2
Research stages
The research is divided into several stages (directly relating to the Output
in Figure 2):
1. Literature review
2. Expert’s point of view
3. Develop CRP solution(s), documentation and templates
4. Evaluate CRP solution(s)
5. Scientific papers and master thesis finalization
7
Due to the cyclic nature research process, any of the six stages can be revisited. For example, it is essential to design research that the product is
validated and improved over and over again. As specified by Hevner et al.,
in order to make the best of a DSRIS the guidelines of Table 1 must be
followed stricktly.
Design-Science Research Guidelines
Guideline
Guideline 1: Design as an Artifact
Description
Design-science research must produce a viable
artifact in the form of a construct, a model, a
method, or an instantiation.
The objective of design-science research is to
develop technology-based solutions to
important and relevant business problems.
The utility, quality, and efficacy of a design
artifact must be rigorously demonstrated via
well-executed evaluation methods.
Effective design-science research must provide
clear and verifiable contributions in the areas
of the design artifact, design foundations,
and/or design methodologies.
Design-science research relies upon the
application of rigorous methods in both the
construction and evaluation of the design
artifact.
The search for an effective artifact requires
utilizing available means to reach desired ends
while satisfying laws in the problem
environment.
Design-science research must be presented
effectively both to technology-oriented as well
as management-oriented audiences.
Guideline 2: Problem Relevance
Guideline 3: Design Evaluation
Guideline 4: Research Contributions
Guideline 5: Research Rigor
Guideline 6: Design as a Search Process
Guideline 7: Communication of Research
'
Table 1: Design Science Research in Information Systems guidelines as proposed by Hevner et. al.
A detailed explanation and justification of the six main stages of this research
follows.
2.2.1
Stage I: Literature review
Since there is an apparent gap in the academic literature on requirements
prioritization in the gaming industry, the aim of this stage will be to identify the popular/established requirements prioritization techniques in regular software development. Moreover, it is quite important to analyze their
features and point out why these solutions cannot adequately address the
specificities of the game development.
The other side of the literature review aims at setting a manageable, unambiguous definition of creativity. Looking for a definition, the omnipresence
8
!"#$%&'()%#*#'+&,-,#".'/'0.%1#"23"&'4)%&5%21,5'/'66789:9'
of term creativity should be kept in mind and identify the domains to be explored. For this research, we look at literature in the domains of philosophy
and history of art. The gained knowledge will be used in the creation of the
Creative Requirements Prioritization technique.
2.2.2
Stage II: Expert’s point of view
At this stage of the research, the focus is on gathering as much in-depth
information on the common practice of requirements prioritization in the
gaming industry as possible. This will be achieved through semi-structured
interviews (investigation of the common practice when it comes to requirements prioritization) and dialogs with experts in the field – game developers,
project managers, software product managers, designers etc.
On the other side of this stage is the investigation and reflection of the industry view on the role of creativity in entertainment software. Additionally, we
look for personal ideas and suggestions as to how can creativity be reflected
in requirements prioritization.
By using semi-structured interviews and open-question surveys1 , we allow
for a more informal knowledge gathering process where the respondents can
naturally steer the direction of the answers to where they feel most comfortable, i.e. their field of expertise.
2.2.3
Stage III: Develop a CRP solution
Using the knowledge gained in stage I and II as a starting point, the creation
of a solution that takes into account the creative aspects of game development is created. This is achieved by distilling the core features of creativity
in entertainment software, combined with the identified needs of the industry. Furthermore, during this process, knowledge gained from available software product management literature on requirements prioritization should
be taken into account - i.e. choosing appropriate requirements prioritization
technique(s) or (or separate elements) to be modified and/or combined in
the creation of a base for the developed solution(s).
1
The interview and questionnaire protocols are discussed later on in section 3 Inquiries
rationale and strategy.
9
2.2.4
Stage IV: Evaluation
Ideally, at this stage, the requirements prioritization solution(s) developed
enters a cycle of validation and refinement. During the validation process,
the designed solutions are validated by industry experts using real world
test data. Despite our effort, our solution could not be tested in a gaming
company, since smaller companies did not have a project to test it on and
bigger studios had issues with sharing information about upcoming projects.
Thus, at this stage, CRP undergoes evaluation, whereas validation (test in
real world) is left for future work.
The proposed models/solutions are evaluated by two types of industry experts – one have been following the progress of development and refinement,
and the other see it for the first time. By doing so, the latter type of experts
provide unbiased feedback, which reveals inherent problems that would have
otherwise been missed.
2.2.5
Stage V: Documentation
In order to make the produced solutions available to people who are not
familiar with the techniques at hand, a set of documentation explaining the
exactly how and in what situations to apply. Moreover, the documentation that goes with any technique produced in this research will explain the
rationale behind developing the specific situation-tailored solution.
2.3
Formalized overview
This section presents the reader with a detailed and formalized description
of research approach, organizing and modeling the notions until now. The
modeling is done using a meta-modeling technique first proposed by Saeki
[67] and later updated and adjusted by van de Weerd, Brinkkemper, Souer
and Versendaal [82]. The technique, Process-Deliverable Diagram (PDD) is
named after its two core modeling foci – meta-modeling of processes and
their respective deliverables. PDD is a combination of UML’s activity and
class diagraming, as proposed by the Object Management Group [53]. In
short, a PDD is a two-sided diagram, representing the processes modeled on
the left, and their respective deliverables/concepts on the right. Figure 3
illustrates a detailed overview of the research approach followed.
10
RELEVANT LITERATURE
Literature exploration
1..*
Study popular requirements
prioritization techniques
RELEVANT RP
TECHNIQUES LITERATURE
d
is based on
Study literature on
creativity
RELEVANT CREATIVITY
LITERATURE
0..*
Researcher
RESEARCH QUESTION
1
1..*
Identify expert-relevant
questions
Expert’s point of view
exploration
are based on
QUESTION TO EXPERT
Researcher
EXPERT INQUIRY
Create interview
protocol
Create survey
Question ID
Question
Researcher
1
Conduct interviews
is based on
Send out surveys
is based on
Researcher & experts
1
Get inquiry results
EXPERT INQUIRY RESULT
Researcher
Data analysis
Analyze empirical
data
Analyze literature
exploration ndings
EMPIRICAL DATA
De ne creative element
in entertainment software
is based on
De ne creative element e ects
on entertainment software
Create potential creative
requirements prioritization
features list
POTENTIAL CRP FEATURE LIST
1..*
Researcher
is based on
Solution prototyping
1..*
Propose prototype
solution(s)
PROTOTYPE SOLUTION
Researcher
[else]
1
Prototype name
Version
Validate prototype(s)
1
Expert
0..*
has
has
TEMPLATE
[solution approved]
0..*
Finalize proposed
solution(s)
DOCUMENTATION
Researcher
Figure 3: Detailed research approach illustrated using a Process-Deliverable
Diagram
11
The following tables complement the research approach PDD by elaborating
the activities (see Table 3 ) and concepts (see Table 4 ) as specified by the
method engineering approach specified by van de Weerd et al. [82].
Research Approach – Activities table
Activity
Sub-activity
Literature
Study popular
exploration requirements prioritization
techniques
Study literature on
creativity
Expert’s
Identify expert-relevant
point of
questions
view
exploration Create interview protocol
Create survey
Conduct interviews
Send out surveys
Get inquiry results
Data
analysis
Analyze literature
exploration findings
Analyze empirical data
Solution
prototyping
Define creative element in
entertainment software
Define creative element
effects on entertainment
software
Create potential creative
requirements prioritization
features list
Propose prototype
solution(s)
Validate prototype(s)
Finalize proposed solution(s)
Description
Study and reviews available and relevant
requirements prioritization techniques.
Look for definitions of creativity and study
creativity’s features.
Compose a list of questions aiming at eliciting the
expert’s point of view on the research-relevant
subjects.
Prepare an interview protocol, based on the list of
expert-relevant questions in the appropriate order
with background information (where required).
Create a survey, based on the list of expertrelevant questions in the appropriate order with
background information (where required).
Conduct the interviews, following the interview
protocol.
Send out the surveys to the experts that are
unavailable for person-to-person interviews.
Collect survey results and summarize the personto-person interview findings.
Analyze the requirements prioritization algorithms
and concepts found in the related academic
literature and express their defining features.
Analyze the collected empirical data from the
surveys and interviews conducted and summarize
it.
Formulated the creative element of entertainment
software, relative to the research at hand.
Define the features of creativity in respect to its
role in entertainment software and their effects on
the development process.
Sift out the features and characteristics that will
potentially help create a creative requirements
prioritization solution.
Using fragments and features from the
POTENTIAL CRP FEATURES LIST, create a
prototype requirements prioritization
algorithm/solution facilitating the gaming industry
needs.
Get expert feedback on the proposed solution.
When/if solution is approved by experts, finalize it
by creating the appropriate documentation and
templates.
!
Table
2: Research approach activities as seen in the PDD (see Figure 3).
!
In Table 3, the concepts and deliverables related to the activities and sub12
!
activities during the different research phases are described.
Research Approach – Concepts table
Concept
Description
Relevant literature
A collection of all the literature related to the domain of the
research.
Relevant RP techniques
A subset of the relevant literature regarding the requirements
literature
prioritization techniques.
Relevant creativity
A subset of relevant literature on creativity.
literature
Research question
The research question is the pivotal structural fragment
around which the research revolves and is based on the
relevant literature. This concept is not connected to any of the
activities in the research approach as seen in the PDD. The
reason for this is that the question was formed prior to starting
the research and is out of the diagram.
Question to expert
A question aiming at exploring the RP experience and/or
ideas regarding creativity’s role in entertainment software from
a game development industry expert.
Expert inquiry
The set of expert-relevant questions present in the interview
protocol and survey exploring the expert POV.
Expert inquiry result
A summary of an expert POV interview and/or survey.
Empirical data
A collection of the summaries of all interviews and surveys of
experts.
Potential CRP feature list
A list of potential creative requirements prioritization features
and characteristics to be used in the solution prototyping
phase.
Prototype solution
A solution prototype based on the empirical data and the
potential CRP feature list aiming at improving the
requirements prioritization process in game development.
Made up of method fragments – coherent pieces of
information system development methods as specified by
Brinkkemper (1996).
Template
A template/outline/guide complementing the solution
prototype.
Documentation
All the required documentation complementing the solution
prototype.
'
Table 3: Research approach concepts as seen in the PDD (see Figure 3).
Master'Thesis'Proposal'|'Aleksandar'Chervenkov'|'3318060'
13
3
Inquiries rationale and strategy
Up until now, attention was paid to the general structure of the research approach. The focus was on what is being researched. There is no doubt knowing exactly what one is looking for in a research is of upmost importance,
but it is now time to look into the how. This section puts the interviews,
questionnaires and the questions in them under the spotlight. A detailed
overview of the questions and their formation, as well as the reasoning behind the choices made follows.
3.1
Strategy
In Greek, the word strategos means “the thinking and action of a general.”
Despite the fact that there is no actual war going on here, the project at hand
battles a complex and volatile problem that needs a general way of perceiving
it. In other words, a well-conceived strategy that provides a framework for
decision-making and action is paramount in order for seemingly isolated tasks
and activities to fit together, integrating separate efforts towards a common
purpose.
In his book Qualitative research and evaluation methods, Patton [55] presents
12 principles of qualitative inquiry – a comprehensive and coherent strategic framework for qualitative inquiry, including fundamental assumptions
and epistemological ideals. These 12 strategies fall into three main themes
of qualitative inquiry – Design Strategies, Data Collection and Fieldwork
Strategies and Analysis Strategies. For the purpose of this particular research, a strategy from each qualitative inquiry theme is adopted.
3.1.1
Design strategy
In Naturalistic inquiry [44], the authors compare extensively the design characteristics of qualitative/naturalistic inquiry versus quantitative/experimental
methods and conclude:
“What these considerations add up to is that the design of a naturalistic inquiry (whether
research, evaluation, or policy analysis) cannot be given in advance; it must emerge,
develop, unfold. . . The call for an emergent of the by naturalists is not simply an effort
on their part to get around the “hard thinking” that is supposed to precede an inquiry;
14
the desire to permit events to unfold is not merely a way of rationalizing what is at
bottom “sloppy inquiry”. The design specifications of the conventional paradigm form a
procrustean bed of such a nature as to make it impossible for naturalist to lie in it — not
only uncomfortably, but at all.”
– Lincoln and Guba [44]
please exph relies heavily on naturalistic inquiry and the reason for this is
partially the explorative nature of this venture. Going deep into unknown
waters hardly allows for a specified and detailed in advance plan. However, a
strategy is needed, and that’s where Patton’s work [55] steps in. He proposes
3 different design strategies of which one facilitates the flexibility expected
in an explorative research as this one. It is called Emergent design flexibility
and is defined as follows:
? Emergent design flexibility - Openness to adapting inquiry as understanding deepens and/or situations change; the researcher avoids
getting locked into rigid designs that eliminate responsiveness and pursues new paths of discovery as they emerge.
Adapting Emergent design flexibility as a design strategy permits the research to naturally explore the domain, allowing for the inevitable change
in; (2) standing the problem and adequately and flexibly adjusting the proposed solutions.
3.1.2
Data collection and fieldwork strategy
Another important design issue is data collection. Deciding between naturalistic inquiry and experimental approach is a design issue. Controlled
experimental designs predominantly aim at statistical analyses of quantitative data, while qualitative data is primary focus in naturalistic inquiry.
This research is no different. The data collected here is largely descriptive
and highly subjective. The insights collected communicate someone’s (i.e.
field expert’s) experience and its derivative conclusions. The data is telling
a story, and as every story has a different reading, the researcher needs to
employ a strategy to help sustain an objective point of view as much as
possible.
In a situation as this one, a serious issue is picking the right cognitive and
emotional stance towards the people inquired and their problems. Patton [55]
15
introduces 4 data collection and fieldwork strategies of which one emerges
as the right choice for the framework of this research — namely Empathic
neutrality.
? Empathic neutrality and mindfulness - An empathic stance in interviewing seeks vicarious understanding without judgment (neutrality)
by showing openness, sensitivity, respect, awareness, and responsiveness; in observation it means being fully present (mindfulness).
As Patton [55] points out, such a strategic thinking helps as there is no
universal perception that can capture the range of possibilities since the
answers will depend on the situation, the nature of the inquiry, and the
perspective of the researcher. This strategy is a way of reaching a middle
ground between becoming too involved, which might cloud judgment, and
remaining too distant, which can reduce understanding.
3.1.3
The pitfalls of qualitative/naturalistic inquiry
Regardless of the strategy employed, a sound research ultimately requires
credibility to be useful. The need for unbiased, objective, honest, credible,
and empirically backed findings is paramount in any respectable scientific
work. All those qualities depend on the researcher’s ability to take and
maintain a position of neutrality [55] with regard to the phenomenon under
study.
Patton [55] notes obtaining neutrality as one of the most difficult task a
qualitative researcher faces - the strategies he proposes include techniques for
helping the investigator become aware of and deal with selective perception,
personal biases and theoretical predispositions.
In this research, investigation of the problems and situations in the domain
of entertainment software development is done with the maximum efforts in
avoiding any bias. The research does not aim to prove a theory or perspective. The investigator here sets to test (but not prove) theories, with the
desire to understand the problem as it unfolds – with its intricacies and different angles. The ultimate goal is to be as balanced and neutral in reporting
both confirmatory and disconfirming evidence as possible.
To conclude, all the pitfalls identified can be avoided with the proper approach. Even in the case of the explorative qualitative inquiry (i.e. the
16
research at hand), strategic thinking can help aid a scientific rigor and yield
valuable insights on which others can build on.
“The idea of acquiring an “inside” understanding – the actor’s definitions of the situation
– is a powerful central concept for understanding the purpose of qualitative inquiry.”
– Thomas A. Schwandt [70]
3.2
The questioning
This section discusses the design and administration the questionnaire, interview protocol and the questions in them as instruments for the conduction
of the research at hand.
3.2.1
Questionnaire, interview or both?
One of the main issues with the conduction of this research was figuring out
how to reach out to experts in the field and get as much detailed insight on
the problems researched as possible. Initially, the idea was to get to a game
development company and conduct the research there. However, this turned
out to be quite impossible. The problem lied in the general culture of secrecy and closed-door policy surrounding game development. The extremely
competitive behavior of the industry results a very solid wall that makes it
quite difficult to study the internal processes of game development.
There is no question that ideas for entertainment products and the products
already in development need to be protected from the competition. The only
viable workaround to this problem was to reach individual developers who
were willing to participate in this study and interview them or have them fill
out a questionnaire. Since practices of these individuals and the processes
they follow are property of the game studios that employed them (with the
exception of self-employed developers), much of the data provided is under
the cloak of anonymity.
Understandably, when doing an explorative research in a fairly unexplored
domain (with the limitations explained above in mind), the best practice
would be to stick to semi-structured interviews that allow for the muchneeded flexibility. However, there is no reason to limit the inquiry to just
interviews and ignore questionnaires. Ultimately, the decision to employ
17
every possible way to gain insight into game development processes and
practices was made.
3.2.2
Closed vs. open questions
First things first, when talking about creating questions for a questionnaires
one has to choose between closed questions and open questions. Let us touch
on the advantages and disadvantages of both.
Bryman and Bell [17] compose a comprehensive list of advantages and disadvantages of open and closed questions:
Closed questions advantages
? Ease of processing the answers. In the case of a structured interview,
the interviewer can circle/tick the answer for the appropriate response. The
process from then on is almost mechanical.
? Enhanced comparability of answers – making it easier to show the relationship between variables and to make comparison between respondents or
type of respondents.
? Added clarity to the questions. Sometimes, if the respondent is not sure
what the questions is getting at, the availability of answers my help clarify
the situation.
? Ease and speed of completion. Respondents are not expected to write
extensively, but just put a tick/circle the right answer.
? Reduced possibility of variability. If interviewers do not write down
exactly what respondents say to them when answering questions, a source
of bias and hence of invalidity is in prospect. Closed questions reduce this
possibility.
Closed questions disadvantages
? Loss of spontaneity. There is always a chance respondents might come
up with interesting replies that are not covered by the fixed answers provided.
Avoidable if one of the answers is left for the respondent to define (i.e. ‘Other’
category).
? Difficult to make up mutually exclusive fixed answers. It is sometimes difficult to create answers that do not overlap.
18
? Difficult to make up exhaustive fixed answers. All possible answers
have to be covered by the fixed answers. Might end up with excessively long
list of answers.
? Possible variation in the interpretation of answers by respondents
– answer validity in jeopardy.
? Irritating to respondents – when unable to find the category that they feel
applies to them.
? Difficulty establishing rapport – in the case of interviews, interviewer
and respondent are less likely to engage with each other in a conversation.
Although respondents typically prefer closed questions, open questions do
have certain advantages over closed ones. Following are the advantages and
disadvantages of using open questions (i.e. allowing respondents to form
their answer as they see fit) for interviewing/questionnaires as specified by
Bryman and Bell [17]:
Open questions advantages
? No forced answers – respondents answer in their own terms.
? Allow for unusual responses – replies that the survey researcher may not
have contemplated are possible, perhaps avoiding bias.
? Avoiding answer suggestion – respondent’s levels of knowledge and understanding can be tapped. The salience of issues for respondents can also be
explored.
? Useful for exploring new areas – areas in which the researcher has limited
knowledge.
? gaful for generating fixed-choice format answers.
The survey researcher is presented with some serious problems when using
open questions (see below ).
Open questions disadvantages
? Time-consuming – open questions are quite time-consuming for interviewers to administer. Understandably, interviewees are likely to talk for much
longer than is usually the case with comparable closed questions.
19
? Answers need to be ‘coded’ – another time-consuming task, each open
question entails reading through answers multiple times. Once to derive
themes that can be used to form the basis for the codes, and then once more in
order to codify the answers into a computer spreadsheet. Similar to content
analysis. All this introduces a high chance of variability in coding of answers.
? Greater effort from respondents – might not pose a big problem when interviewed, however in the case of self-completion questionnaires, respondents
might be reluctant to write for too long. There is a good chance using open
question questionnaires will result in low response rates.
After careful consideration of the characteristics of both open and closed
questions, a decision to use open questions exclusively was made. The reasoning behind this choice is based, on one hand on the very explorative nature
of the research and on the other – the desire to avoid limiting respondents’
answers.
For this research, it is crucial to be able to expand the understanding of
problem at hand as deep and broad as possible. Even though the response
rate of the questionnaire was (as expected) very low, the alternative would
have contradicted with the explorative goal of this research.
20
4
Theoretical foundations
Proceeding to the first stage the research (see 2.2 Research stages) it is now
time to look into the literature foundations of our research.
The research scope diagram from section 1.2 illustrated a layered approach
to exploring the different research concepts involved in reaching the research
goal. To recap, the diagram illustrated three core emphases of the research
goal (which can also be seen in the right side of Figure 4 beneath):
(1) understand the creative element of entertainment software;
(2) understand how this creative element affects development in regards to
requirements prioritization; and finally
(3) use the knowledge in developing a requirements prioritization solution
tailored in the specificities of the game development domain.
However, before tackling these problems, there is the outer layer of research
concepts that needs to be systematically “peeled off” first.
In order to (1) understand the creative element of entertainment software,
(1)
en
te
r
nt in entertainm
me
en
ele
t
(3)
n RP
*
Creative
requirements
prioritization
te
n iq
ee
le m ent effects
on
(2)
ue
s
iv
Creat
po
RP
tiv
up
*
ch
ts
Cr
RP
ea
rt i
*
Cre
ati
ve
Def
ini
tio
n
Creativ
ity i
n
ftware
t so
en
inm
ta
ity
ativ
e
r
c
of
ity
ativ
cre
of
are
ftw
so
Def
ini
tio
n
we need to start by defining creativity.
ee
le m
en
* requirements
prioritization
Figure 4: Outermost layer of the research scope diagram: define creativity.
21
4.1
The creative prespective
This section looks into literature on the complex phenomenon creativity;
history and definition of the word itself; and academic work done trying to
better understand the mechanics behind it. Ultimately, the goal is to see how
creativity fits in the context of entertainment software (i. e. video games).
4.1.1
Definition of creativity
The earliest appearance of the English word “create” dates back as early as
the late 1300’s [76, 37]. The poet Geoffrey Chaucer, commonly referred to
as the father of English literature used the word “create” in his long and
unrelieved prose treatise on virtuous living – The Parson’s Tale 2 .
It was not until the Enlightenment that the modern meaning of the word (i.e.
the act of human creation) emerged [76]. Words have no uniform definitions
and a word as ambiguous as creativity has as many definitions as there are
dictionaries. For example, the online Oxford Dictionary lists the following
definition:
creativity |kri:eI"tIvIti|
noun
the use of imagination or original ideas to create something; inventiveness:
e.g firms are keen to encourage creativity
Te above is a very simple definition that would help anyone with no previous
knowledge of the word understand the meaning and use it in a sentence.
However, this definition is far from comprehensive and this research requires
an unambiguous complete definition. A scientific take on defining creativity
should look like that:
2
"And eke Job saith, that in hell is no order of rule. And albeit that God hath
created all things in right order, and nothing without order, but all things be ordered
and numbered, yet nevertheless they that be damned be not in order, nor hold no order."
– excerpt of The Parson’s Tale by Geoffrey Chaucer
22
Creativity is the ability to produce work that is both novel (i.e., original, unexpected) and appropriate (ie., useful, adaptive concerning task constraints). [76]
The scientific definition breaks down the meaning into the word’s specific
features leaving no ambiguities. Coming up with a good definition is an important step to understanding creativity. Nevertheless, understanding how
the phenomenon creativity actually works requires extensive research.
4.1.2
Research on creativity
The academic society has largely neglected this almost mystified word “creativity” until the second half of the twentieth century. The American Psychological Association archive reveals that since the foundation of the association in the end of 19th century until about 60 years later, psychologists
paid little-to-no attention to the phenomenon. Fast-forward to 1950 and
the American Psychological Association Presidential Address where Guilford [30] brings the attention of researchers to the problem and challenges
the research society to pay closer attention to creativity. Research interest
in the topic remained marginal in the 50’s with interest tripling by the mid
90’s, yet still barely reaching 0.5% of psychology articles published [37].
Interest in the systematic study of creativity has been growing substantially for the past few decades, leading to some radical advances in our
understanding of the nature and implications of creativity [50]. Guilford’s
infamous divergent thinking test method laid the basis for studying creativity (for the first time ever) as an empirical phenomenon. Despite criticism
[87], Guilford’s work remains important in contemporary creativity research
[9, 58, 38]. However, as Mumford [50] points out, a phenomenon as complex
as creativity should not be studied relying solely on one method. Fortunately, others have proposed their own takes on understanding creativity.
Mark Runco’s Creativity Research Handbook [64] and Robert Sternberg’s
Handbook of Creativity [76] summarize recent work on creativity research,
devoting chapters to different methods such as Simonton’s historiometric approach [72, 73], the case study approach advocated by Gruber and Wallace
[29] and Policastro and Gardner [59], Csikszentmihalyi’s systems model [21],
neuroscience studies of the type described by Katz [36] and Martindale [47],
as well as the cognitive and computer modeling studies described in chapters
23
by Boden [14]; Finke [24]; Ward, Smith, and Finke [80]; and Weisberg [83].
All those researchers and their work offer a different angle and an insight
into understanding creativity, however there is one theory that can be particularly useful in the process of understanding creativity in the context of
entertainment software - Csikszentmihalyi’s A Systems View of Creativity
[21]. The following section elaborates on Csikszentmihalyi’s work and more
specifically his Systems Model of Creativity and its relation to this research.
4.1.3
Csikszentmihalyi’s Systems Model of Creativity
Mihaly Csikszentmihalyi’s take on creativity differs from the general understanding quite substantially. Many have noted, there is a tendency to see
creativity exclusively as a fundamentally self-expressive mental process that
is free from any discernable constraints [88, 57, 81, 68]. Csikszentmihalyi
on the other hand believes that besides the individual, there are two salient
aspects that need to be taken into account: the cultural or symbolic aspect
with its rules called the domain; and the social aspect called the field. He
proposed a theory of creativity that asserts creativity is the result of a very
active relation between these three separate components in a system - what
he calls a Systems View of Creativity [21]. The theory is expressed with the
help of his Systems Model of Creativity seen in Figure 5.
24
Henry-3442-01.qxd
7/12/2006
6:01 PM
Page 4
Csikszentmihalyi
Cultural
System
DOMAIN
(knowledge,
tools, values,
practices)
Transmits the existing
body of knowledge
Evaluates innovations &
retains selected ones
PERSON
(individual
practitioner)
FIELD
(community of
practice,
gatekeepers)
Produces
innovations
Social
System
Genetic
makeup,
talents,
experience
Figure 1.1
systems model of creativity
Figure 5:AMihaly
Csikszentmihalyi’s Systems Model of Creativity [21]
As previously stated, the model (see Figure 5 ) illustrates the interdepensymbolisation
thatthe
made
it possible
insights tothree
be shared
and evaluated
by
dences
between
Systems
Viewforoftheir
Creativity’s
elements
– a culture
othersits
who
had equivalent
with
symbolic
rules; atraining.
person who brings a novel product to the domain;
But most novel ideas will be quickly forgotten. Changes are not adopted unless they
are sanctioned by some group entitled to make decisions as to what should or should
interconnections
shown
in the
model
as ‘dynamic
ofhere
circular
causalnot be included in the
domain.
These
gatekeepers
are whatlinks
we call
the field.
Here
field [19]
refersofonly
to the
social organisation
of the
domain
– to the
jourity’
equal
importance,
hence the
starting
point
on teachers,
this mapcritics,
is purely
nal editors, museum curators, agency directors, and foundation officers who decide
arbitrary.
what belongs to a domain and what does not. In physics, the opinion of a very small
In
orderofforleading
creativity
to occur,
first awas
setenough
of domain-specific
and pracnumber
university
professors
to certify thatrules
Einstein’s
ideas
were creative.
of millions
people accepted
the judgement
of thisneeds
tiny field
tices
need toHundreds
be adopted
by theof individual.
Next,
the individual
to
and marvelled at Einstein’s creativity without understanding what it was all about. It
produce a novel variation entry to the content of the domain. Finally, the
has been said that in the United States 10,000 people in Manhattan constitute the
variation
needs to be accepted and included in the domain of similar work.
field in modern art. They decide which new paintings or sculptures deserve to be seen,
bought,
included
collections,
therefore added
to theto
domain.
The
reason
why in
this
model is and
particularly
interesting
the research at hand
and field experts who validate the innovation [20]. The author explains the
is that it easily translates to the process of developing creative products.
Take for example the case of video game development where a person (e.g.
4
25
game designer) comes up with a game idea that he/she believes is novel
compared to other video games (i.e. the field ) and is recognized as novel (or
en
te
r
nt in entertainm
me
en
ele
t
*
ftware
t so
en
inm
ta
n RP
(3)
Creative
requirements
prioritization
te
n iq
RP
ee
le m ent effects
on
(2)
ue
s
Creat
po
tiv
up
*
ch
ts
Cr
RP
ea
rt i
*
Cre
ati
ve
Creativ
ity i
n
(1)
ftware
t so
en
inm
ta
en
te
r
ity
ativ
cre
of
are
ftw
so
Creativ
ity i
n
Def
ini
tio
n
alternatively is rejected) by the community (i.e. the domain).
iv e
ele
m
en
* requirements
prioritization
Figure 6: Outermost layer of the research scope diagram: creativity in enterainment software.
The Systems Model of Creativity offers a way of studying the creative processes of game development not by focusing on the individual (e.g. the game
designer) but as viewing the phenomenon as a system considering the impact
of all stakeholders involved. While the model has had its critiques [84, 60],
it serves as a logical way to look at game development especially in the game
studio environment. The model can be used to explain creativity on multiple
levels. Applying the model on the highest level (industry level) would look
like that:
26
vi
et
me
o ga mark
de
gam
io / deve
tud
lo
es
PERSON
FIELD
r(s)
pe
ers
tom / user
us
s
c
DOMAIN
Figure 5a: Applying Csikszentmihalyi’s Systems Model of Creativity on
game industry level
The interaction between the three system components is quite straightforward. A game developer(s) or game development studio (PERSON) delivers a game to the market (DOMAIN). A video game might be considered
great by its developers, but it is the FIELD that will evaluate it and decide
if it will be played or even remembered.
In the case of large entertainment software studios for example, Csikszentmihalyi’s model can also easily be translated to model the creative process
on a more specific level (project level) and stage during development:
27
g
s
e develope
r
am
DOMAIN
me testers
ga
es
me d igner
ga
FIELD
PERSON
Figure 5b: Applying Csikszentmihalyi’s Systems Model of Creativity on
project level
In this case, the interaction between the system components is as follows:
a game designer (PERSON) comes up with a design for a new game and
shares it with the game developers (DOMAIN) who in tern use specific
knowledge, tools, practices and values to produce a version of the game.
Once in a state at which the game can be played, it is tested by the game
testers (FIELD) who provide feedback/evaluate the game.
As we have shown, the variety of applications of the Systems Model of Creativity and the model’s flexible nature and relative generality offers convenient means of modeling creativity in entertainment software.
4.1.4
Creative element in entertainment software
In order to understand the role of the creative element in video games and
it’s effects on video game production (see Figure 7 ) it was imperative to set
a scientifically sound definition of creativity itself (see 4.1.1 Definition and
Figure 4 ) and understand the mechanics behind the creative process (see
4.1.3 Csikszentmihalyi’s Systems Model of Creativity and Figure 6 ).
28
e
*
n RP
on
R
C re a t
po
le m ent effects
(2)
es
rt i
P*
ee
up
*t
qu
tiv
ts
Cre
ati
ve
Defi
nit
ion
Cr
RP
ni
en
te
rt
are
ftw
so
(3)
Creative
requirements
prioritization
are
ftw
so
Cre
ati
ve
n
nt in e tertainm
en
me
t
ele
ea
ch
Creativ
ity i
n
(1)
are
oftw
nt s
me
ain
(1)
n
nt in e tertainm
en
me
t
ele
vity
ati
cre
of
i ve
ele
m
en
* requirements
prioritization
Figure 7: Inner layer of the research scope diagram: creative element in
enterainment software.
With the above in mind,
the creative element in entertainment software is a multifaceted concept that intertwines the developer’s personal capacity to produce novel work and recognize its
appropriateness with his/her interpersonal abilities to
utilize domain knowledge, values, tools and best practices.
The creative element and all relevant concepts and models discussed in this
section will be discussed further when designing the Creative Requirements
Prioritization solution.
4.2
The technical prespective
In this section of the literature review, the focus is shifted to the technical side of this project. In order to reach the ultimate goal of designing a
requirements prioritization solution suited for entertainment software production, we continue following the process by studying relevant scientific
29
literature on the broader subject of game development. Moreover, some of
the more relevant software development techniques and requirements prioritization methods will be assessed with regards to their relevance to the
project at hand.
4.2.1
A brief history of video games
It is debatable which is the first video game and this information is of little
importance to the project at hand. However it is still interesting to see how
far video games have come since their inception about 60 years ago. In the
distant 1952 at the University of Cambridge, Alexander S. Douglas created
OXO3 – a digital version of tic-tac-toe - the first computer game to use a digital display [49], followed by Tennis for Two4 in 1958 – a game developed by
Willy Higinbotham, a physicist working at Brookhaven National Laboratory.
These two games perfectly illustrate how innocent, if you will, video games
were in the beginning. There were no rules as to what hardware, what software is used, much less what processes or methodologies should be followed
in developing video games. However all that was hardly a problem as these
games were dead simple compared to modern 20-million-budget-games, and
more importantly – non-commercial.
3
An interesting fact about OXO was that human input was provided using rotary phone
interface.
4
Tennis for Two’s graphic user interface uses an oscilloscope (see Figure 8).
30
Figure 8: Tennis for Two - (arguably) the second video game ever. Created
by Willy Higinbotham in 1958 at Brookhaven National Laboratory
It was not until the 1970s that the first commercially designed video games
emerged, running either on arcade game machines or first-generation consoles. The first ever commercially designed game to reach the public was
Computer Space – a coin-operated arcade released in 1971. One year later
the iconic Pong was already available and perhaps singlehandedly formed
the video game industry [4].
Pong was such a hit, that the competition delivered an avalanche clones,
resulting to the inevitable video game crash of 1977 when many manufacturers of older consoles and Pong clones cleared stock at a loss and exited
the market. The lack of diversity of and originality of games almost killed
the industry. However, Taito’s Space Invaders in 1978 not only revived the
industry, but also marked the beginning of the golden age of video arcade
games by inspiring developers to break out of the box (i.e. Pong) [85].
By 1982 the annual revenue generated by arcade game machines had grown
to $8 billion. At the time, home console market was benefiting from the popularity of video games and approximately 8 million American homes owned
a console, generating annual revenue of almost half that of the publicallyavailable coin-operated arcade games [63].
By mid-1980s personal computers led to the emergence of hobbyist game development. Initially, taking advantage of the weak copyright regulation, hob31
byists would just type the source-code of popular games into their machines,
which was available in specialized magazines and books. Unsurprisingly,
Atari were displeased and raised the issue of software piracy and copyright
infringement on behalf of both users and the competition [46].
Aside from allowing software piracy, the relatively cheap publishing costs and
the growing base of professional and hobbyist game development allowed for
bold, unique video games to emerge. This creative peak of video games in
the mid-1980s defined many of the genres that are still popular today.
Fluctuating success of the industry continued, but what got stronger with
the years is the culture of video games – many users began to expect creativity and originality, which has been getting harder to deliver with the
growing number of games different games. Moreover, with the sky-rocketing
budgets popular modern PC games generally have, failing is not an option,
thus game studios often invest in the safer remakes of successful titles. The
recent popularization of mobile gaming platforms combined with new business models born in recession times, opened the window for a fresh breeze
of original and creative, inexpensive to develop and to buy games – echoing
the success of the golden age of video games in the 1980s.
Regardless of the originality of games, the underlining development process
is a creative one. In other words, the concepts that Csikszentmihalyi talks
about (see 4.1.3 Csikszentmihalyi’s Systems Model of Creativity) can be seen
whether trying to emulate the formula of an already existing game(s) or produce a novel game idea. We argue that having a good understanding of the
separate components in Csikszentmihalyi’s model and their interdependencies, and using that knowledge can help increase the quality and success rate
of entertainment software. However, introduction of these concepts to the already established development techniques should be done in a non-intrusive,
logical and simple manner as not to introduce any significant overhead. In
the following subsections of the literature review, focus is shifted to game
development to review and see how it could accommodate a more creative
perspective.
4.2.2
Overview of video game development
The process of video game development differs immensely depending on ambition and resources. Whether fueled by desire to develop a great game or
32
to make money (ideally both), ambition is needed to give the initial spark of
any development process. On the tangible side of things, the development
process requires a certain amount of resources depending on the ambition.
Much like DVD/Blu-ray publishers, video game studios/publishers are large
corporations responsible for funding the development process of their products and all aspects of marketing it. Game studios can and do sponsor some
quite ambitious projects, in fact, the average development budget for a crossplatform video game in 2010 was between US$18 - US$28 million, with the
most ambitious projects exceeding US$40 million [5].
Success of a video game does not necessarily go hand in hand with huge
budget. In fact, the only thing a big budget is sure to deliver is higher risk.
There are however, many successful independent games that rely on cheap
Internet distribution schemes for both personal computers and consoles [48].
There are independent game (or indie game) projects that became successful5 regardless of their low budget.
Due to their lower budget and casual/ad-hoc nature of development, independent game projects rarely follow a certain development process. Rules
and hierarchies are more suitable for complex projects with numerous people
working on them. Such a project in a game studio will have different people
working of different tasks. Here are the common roles in a common game
development team [49, 54, 10, 56]:
Roles in a development team
Designer – responsible for conceiving and designing the game, the game designer
is an important member of the development team. In bigger projects it is
common to have a design team led by a lead designer who manages and coordinates the other designers. It is the lead designer’s responsibility to assure
the use of a uniform design language across different parts of the game (i.e.
game mechanics, user interface etc.).
Visual artist – responsible for creating the visual art (2D and 3D models and
graphics, textures, images, background/terrain images, user-interface graphics, animation, cinematic, concept art etc.). In more complex projects, teams
5
It is quite baffling how popular some independent games get. In 2009, Minecraft, a
simple sandbox-building game originally created by Markus “Notch” Persson in just 7 days!
Persson estimated his game made US$33 million by April 2011. MineCon is Minecraft’s
convention. [3]
33
of artists are manages by an art director to assure uniform style and quality
aesthetics.
Programmer – responsible for the project’s codebase, these software engineers
have numerous responsibilities. More complex projects have quite a lot of
programming responsibilities – physics, artificial intelligence, gameplay, user
interface, graphics, scripting, sound etc. These tasks often have separate
teams of programmers with one or more lead programmers who implement the
initial codebase and oversee the development process. Apart from developing
the game itself, sometimes programmers build development tools to help others
in the game development project.
Level designer – responsible to create game levels (i.e. challenge, tasks, missions etc.) for the game using specific software that was either purchased
or developed in-house. Projects with higher budgets will usually use in-house
developed comprehensive level editors/builders 6 to create levels for both the
complete and early/prototype/beta versions of the game.
Sound engineer – responsible for sound effects, soundtrack, voice acting, sound
positioning. Creating all the sound assets for high profile games is quite
expensive and sometimes even bigger game studios cannot afford to do it all
in-house.
Tester – responsible for testing and quality control of the video game since the
very first playable version. The role generally requires discovery and documentation of bugs and inconsistencies as well as general feedback on the
game.
6
These tools generally differ from level editors available to players of the finished game
(if such are made available in the first place). Development-level editors might allow for
the creation of levels regardless of artwork and textures availability at any given stage of
the development process.
34
Game development overview
In their paperDeveloping
Developing
a Reference
Gameby
Production
by Method11
a Reference
MethodMethod
for Game for
Production
Method Comparison
Comparison [79], Inge van de Weerd, Stefan de Weerd and Sjaak Brinkkem-
per developed
reference
method
game production
illustrates
everyand
parametersa are
identified,
which for
comprises,
among others,that
the budget,
recourses
analysis.
detailed
game
concept
is defined
and presented
by means of
actioncompetitive
and deliverable
in A
the
typical
game
production
process.
The complete
Then, in the pre-production phase, the game design document is
modelprototypes.
is remarkably
detailed, comprising of four phases: Concept phase,
developed, which described the story, gameplay and requirements. Also, a project
Pre-production
phase,
Production
phase
Post-production
phase.
plan and staffing
plan
is created. This
lateand
moment
for project planning
and The
staffing
has method
to do with
the nature
of theactivities
game production
process.
Rather which
than the
reference
consists
of 13 main
and 69 sub
activities,
straightforward development process from requirements to implementation that is
result used
in 93indeliverables.
the information systems domain, first the game concept needs to be
approved.
this game
concept isoutline
a creative
with the
For the
purposeDeveloping
of presenting
a digestable
of aactivity,
typicalcomparable
game produc-
production of movies. Someone has to approve or invest in the idea that is developed
in this phase, before you can think about a project and staffing plan.
tion, Figure 9 gives a high-level overview of the process:
Fig. 3. Reference Method for Game Production: Process Overview
Figure 9: Refrence Method for Game Production: Process Overview [79]
Next, in the production phase, the game is implemented in a working game; the
final version. During the implementation scope changes are managed. At the end of
The phase
that our
requirements
prioritization
aids
is pre-production.
this phase,
a demo
is developed
for marketingsolution
purposes.
Finally,
the fourth phase
comprises
the
typical
post-production
activities.
The
game
needs
to
be
As explained by the authors [79], in the pre-production phase, the localized
game de-so it
can be released in different countries, QA tests are done and promotion material for
the marketing department is created. Finally, the project concludes with developing a
35the game.
closing kit and releasing and shipping
sign document is developed, which described the story, gameplay and requirements. Also, a project plan and staffing plan is created. This late moment
for project planning and staffing has to do with the nature of the game production process. Rather than the straightforward development process from
requirements to implementation that is used in the information systems domain, first the game concept needs to be approved. Developing this game
concept is a creative activity, comparable with the production of movies.
Someone has to approve or invest in the idea that is developed in this phase,
before you can think about a project and staffing plan. [79]
4.2.3
Agile development: Scrum
Typical life cycle development methods (e.g. Waterfall) as well as formal
software development methods are not favored in the video game industry
due to their inherent rigidity [13]. Any software project’s success however,
depends heavily on following a good development methodology. Much of the
popular game development literature [13, 10, 49] points to iterative software
prototyping as the definitive instrument to developing video games. One
very common such methodology is agile development. The term agile software development was first introduced in 2001 by the Manifesto for Agile
Software Development. The core values in the manifesto are: “Individuals and interactions over processes and tools; Working software over
comprehensive documentation; Customer collaboration over contract negotiation; Responding to change over following a plan.” [2]
Bates [10] states that most game development projects have no clear requirement outline and as such, benefit from agile development’s dependence on
constant feedback and refinement of the game’s iterations [18]. Chandler [18]
suggests the agile software development method Scrum (see Figure 10 ) as a
particularly popular in the industry, echoing the opinion of game developers
interviewed for this research.
Game developers’ affection to Scrum has to do with the method’s design.
Scrum is designed to be used in an environment where planning ahead is
difficult and requirements change [69]. We will examine in the interworking
of Scrum since the product of this research (i.e. the Creative Requirements
Prioritization method) must to be fully compatible with this prevailing technique.
36
Scrum
24h
7-30 days
Product Backlog
Sprint Backlog
Sprint
Working increment
of the game
Figure 10: Scrum - agile software develepment
Roles in Scrum
In Scrum, there are two types of roles as described by Schwaber [69] – core
and ancillary. (informally referred to as pigs and chickens 7 ). Ancilliary
roles have infrequent and/or limited involvement in the Scrum development
process such as relevant stakeholders and management. Schwaber [69] defines
three core roles - Product Owner, Development Team and Scrum Master.
Product Owner - When working with different stakeholders (clients, testers,
designers, management etc.), the product owner ensures a level of healthy
communication with the Development Team. An exclusive responsibility of
the Product Owner is managing the product backlog (a list of ordered/prioritized
requirements):
? Clearly expressing Product Backlog items;
? Ordering the items in the Product Backlog to best achieve goals and missions;
? Ensuring the value of the work the Development Team performs;
7
In Agile project management, the story The Chicken and the Pig is often used to
provide satirical analogy when talking about commitment to a project. The basic fable is
as follows: Pig and a Chicken are walking down the road. The Chicken says, "Hey Pig, I
was thinking we should open a restaurant!". Pig replies, "Hm, maybe, what would we call
it?". The Chicken responds, "How about ’ham-n-eggs’ ?". The Pig thinks for a moment
and replies, "No thanks. I’d be committed, but you’d only be involved!" [61]
37
? Ensuring that the Product Backlog is visible, transparent, and clear to all,
and shows what the Scrum Team will work on next; and,
? Ensuring the Development Team understands items in the Product Backlog
to the level needed.
Even thought the Product Owner might not accomplish all tasks above
him/herself, he/she remains accountable. A Product Owner could be a member of the Development Team, but Schwaber [69] recommends not to combine
the role with that of the Scrum Master.
Development Team - usually a team of 3-9 people with diverse skill set re-
sponsible for the actual incremental product development (analysis, design,
implementation, testing, technical communication, documentation etc.). The
potentially shippable increments developed during every Sprint – a short development cycle (7-30 days) - are the basic unit of development in Scrum
(see Figure 10 ). Development Teams have the following characteristics:
? They are self-organizing. No one (not even the Scrum Master) tells the Development Team how to turn Product Backlog into Increments of potentially
releasable functionality;
? Development Teams are cross-functional, with all of the skills as a team
necessary to create a product Increment;
? Ensuring the value of the work the Development Team performs;
? Individual Development Team members may have specialized skills and areas
of focus, but accountability belongs to the Development Team as a whole;
and,
? Development Teams do not contain sub-teams dedicated to particular domains like testing or business analysis.
Scrum Master - ensures the Development Team stays focused on the sprint
goals. The Scrum Master does not assume the role of a team leader, but
takes the responsibility of protecting the team of any distractions and impediments that might occur. The role is also responsible for enforcing rules
that help maintaining focus on the established priorities. The Scrum Master
collaborates with the Product Owner in effectively managing the Product
Backlog and clearly communicates the vision and goals to the Development
38
Team. The Scrum Master promotes agile “thinking” to the Development
Team and inside the organization.
Sprint overview
The actual development under Scrum takes place during sprints. Pictured as
the third step in Figure 10, a sprint is the basic development unit in Scrum.
As the diagram shows, a sprint is a cyclic development process that ideally
results in finished, working portions of the software product.
Sprints span no more than one calendar month. The logic behind this rule is
that the longer development takes, the higher the chance its definition might
change. By stopping and looking back at what was done and what is to be
done next, Scrum members can assess and adapt the workload, work pace,
goals, and reflect on the feedback from the previous Sprint, taking appropriate actions. All this is done during Sprint Review and Sprint Retrospective
meetings held at the end of each sprint cycle.
As Schwaber and Sutherland [69] put it, sprints are the heart of Scrum.
However, before getting to the actual development, requirements need to
be defined. This is where Product and Spring Backlogs step in (see Figure
10 ). Simply put, the product backlog is the list of prioritized/ordered list of
requirements defining the software product, where as the sprint backlog is a
“digestible” portion of the requirements from the top of the product backlog,
combined with a detailed plan of action. The spring backlog could be viewed
as the forecast by the Development Team about what functionality will be in
the next Increment and the work needed to deliver it. Both backlogs could
evolve during development as new information about the project emerges.
Documentation on Scrum [69] elaborates on all of those terms and actions
as well as other details, but leaves the question “How to order the product
backlist?” unanswered. With this research project, we provide an answer
to this question in the form of requirements prioritization tailored to the
specificities of video game development.
4.2.4
Overview of typical game development complications
Technically, video games are a type of software and it seems only logical that
their development process should benefit from the same solutions and suffer
the same issues as those found in software product management. However,
39
literature on video game development tells a somewhat different story. According to Flood [25], all post-launch evaluations of game development point
to the same problems: development cannot keep up with the schedule, the
final product contains many defects, implemented functionalities differ from
the original ideas, producing the game took a lot of pressure, and development required an immense number of man-hours to complete.
Flynt and Salem [26] identify the inability to clearly establish the project’s
scope as the biggest reason for project failure. Without a well-established
target, emerging requirements could significantly alter the architecture of
the system, thus causing serious problems. According to the authors, this
commonly occurs when enthusiastic developers experimenting with new, unscheduled features (also known as feature creep) that are thought to make
the game more interesting, but ultimately drive it out of scope and inflate
the size of the project. Moreover, lack of strong requirements analysis and
techniques for improving investigation and verification of the requirements
ensure the existence of this serious problem.
When it comes to scheduling problems, there are few domains that can illustrate them better than the video game industry. Although game projects
usually start with reasonably structured schedules and bright ideas mixed
with a lot of enthusiasm, there is always something that goes wrong, and the
list of things that can go wrong is virtually endless [13]. Often such problems occur due to the extremely diverse disciplines involved in the project,
which generates delays caused by waiting for the work of others, as well as
lack of a realistic estimate in the initial plan, making the team incapable of
fixing a deadline for the project [26]. Moreover, the lack of historical data
and experience with new technologies makes it almost impossible to estimate
deadlines.
In practice, it is natural for a project to change its size and/or complexity.
However, often the planned release date is not moved and what occurs is
what is referred to in basketball as crunch time. In the game industry it is
the equivalent of extreme work overload. Traditional software development
companies undergo such cycles of work overload, however Greshenfield et al.
[28] argue that the game industry goes through such periods more frequently.
Confirmed by Hamann’s [31] experience, working 12-hour-per-day, sevendays-a-week explains the general high turnover rate in the gaming industry.
40
It is true that software is a technologically enabled product, however the
game industry is arguably one of the most technologically advanced software
industries – leaders in graphical computation. Logically, such technologies
frequently involve great effort and a large investment in time [26] and as such
it’s not surprising that a lot of research has focused on methods for game
development.
Summarized, the three most significant problem areas according to game development literature are: scheduling problems, crunch time and technological problems, all of which are directly related to requirements prioritization
and/or other factors.
It would be naïve to look forward to developing a solution that fixes all
problems in the field. The aim is to tackle the problem of requirements
prioritization. Developing a domain-specific prioritization method could improve the development process and lower the risk of the problems discussed
above. The following subsections of the literature review will cover popular
requirements prioritization methods.
41
4.3
Requirements prioritization
This section covers relevant literature on requirements prioritization (see Figure 11 ). At this point of the research, the goal is to look at the characteristics
of popular and relevant prioritization techniques and try and understand how
(3)
*
Creative
requirements
prioritization
te
n iq
ee
le m ent effects
(2)
ue
s
rt i
on
iv
Creat
po
tiv
up
*
ch
ts
Cr
ea
RP
s
RP
te
ue
en
te
r
*
*
n iq
(1)
nt in entertainm
me
en
ele
t
RP
ch
Creativ
ity i
n
n RP
Cre
ati
ve
ity
ativ
cre
of
ftware
t so
en
inm
ta
are
ftw
so
De
nit
ion
effective they are in the context of entertainment software develpment.
ee
le m
en
* requirements
prioritization
Figure 11: Outermost layer of the research scope diagram: requirements prioritization techniques.
Traditional RP techniques exist in three dimensions (or aspects8 ) - cost [52],
value and risk [86]. However, this paradigm might not be entirely appropriate when evaluating requirements for game development. Below is a short
discussion of the these three main aspects.
Risk: In the context of computer graphics and video processing, modernday video games are some of the most advanced pieces of software,
taking advantage of some of the most advanced computer hardware.
This title comes with its own implications to the game development
process: extra risk – one cannot expect to accurately forecast the effort, time and resources when adopting a new technology. Companies
that strive for competitive advantage achieved by adopting the newest
technologies do so because they can afford the risk. Giants like Bliz8
Other more specific aspects of requirements prioritization in literature are: importance [43], penalty [86], time [86] and volatility [39]. However, all those aspects can be
summarized as cost (e.g. penalty, time), value (e.g. importance) and risk (e.g. volatility).
42
zard Entertainment can afford to spend more than a decade developing
a new game9 , however common developers have limited budgets, and
as a result, limited time.
Cost: /absolute or relative value/ As with any other software development
process, estimation of the cost of each requirement in game development is crucial. Among others, the cost of a requirement is influenced by complexity of the requirement, ability to reuse existing code,
amount of testing and documentation needed [86]. The measurement
unit is often expressed as man-hours (or required effort over time). Logically, any development project that has time and/or budget constrains
needs to use an effective cost metric. Game industry’s characteristic
high adoption rate of new technologies discussed earlier, introduces
implications to measuring the cost of such requirements due to the
developer’s inability to estimate required time.
Value/Importance: /absolute or relative value/ Calculating the value (or
importance) of a requirement can be quite confusing. A requirement’s
value is an extremely multifaceted concept since it heavily depends
on the perspective of the stakeholder prioritizing. Lehtola et al. [43]
give: urgency of implementation, added value (or importance) of the
requirement to the product architecture; and strategic importance for
the company as just a few of the possible interpretations of this metric.
Logically, there are as many meanings behind value of requirements as
there are stakeholder interests/views.
As far as requirements prioritization techniques are concerned, the three
broad aspects (value, cost and risk ) discussed above comprise the requirements prioritization arena. There are many techniques, and each one uses
some way of calculating the value, cost and/or risk (and/or possibly their
more specific derivatives) of separate requirements to prioritize them and
help development focus. Naturally, each technique comes with its pros and
cons.
The scope of our research does not stretch to exploration of all requirements
9
Blizzard Entertainment has build a reputation of one the slowest game developers
when it comes to delivering new titles, with CEO’s statements such as “No one will remember if the game is late, only if it’s great”. Development of Diablo 3 started in 2001
and took a full 11 years to complete.
43
prioritization techniques in the field of software development. What is important at this stage of the research is to focus on the popular requirements
techniques and identify any potentially useful prioritization mechanisms to
use in the development of the Creative Requirements Prioritization solution.
4.3.1
Requirements Triage
When developing a complex software product, developers are bound to deal
with an avalanche of requirements. Often when coming up with ideas for a
new video game, developers end up with such a huge list of initial requirements, that it is impossible to fit it in the scope of the project. There is
a very simple method that cuts down the requirements list to an estimated
manageable size considering the available resources of the development team.
It is an old technique called Requirements Triage [22], from the French verb
trier, meaning to separate, sift or select.
Figure 12: Triage - A primitive method of quickly ordering patients by severity of their heath condition. Picture: Triage station, Suippes, France, World
War I.
The technique is an adaptation of the medical term triage - the process
of determining the priority of patients’ treatment based on the severity of
their condition. This method originated during World War I, where French
doctors categorized patients condition as one of four groups that roughly
44
translate to: Can wait, Has to wait, Cannot wait, and Lost. Similarly, Requirements Triage separates requirements to crucial requirements
that must clearly be included; others that should definitely be excluded; and
finally leaves a pool of nice-to-have requirements that need to further be
prioritized. A more advanced prioritization technique to do the actual prioritization is needed at this point. Some of the most popular requirements
prioritization techniques are discussed next.
4.3.2
Analytical Hierarchy Process (AHP)
Unlike the Requirements Triage, AHP is a complex mathematical decisionmaking (or prioritization) technique that is designed to deal with complex
multi-aspect decision problems [6, 66, 65]. AHP is the most commonly referred prioritization technique within decision making in requirements engineering [51] and as such, there is a large of body literature that delves deep
into the interworking and application of AHP. However, the aim here is to
discuss the technique’s relevance to the video game development domain.
The example scenario10 given in Table 4 illustrates AHP being used to find
the relative value/importance of 5 different criteria for choosing a city. In
order to establish the relative value of a single criterion, it needs to be compared to another criterion. The first stage of the process needs the person
responsible for prioritization to compare each criterion to all other criteria.
A criterion can be of higher, lower or equal relative importance relative to
another criterion. This relative importance is represented through a ninelevel scale, where 1 means equal importance between two criteria and 9 is the
maximum difference [66]. These relative values are collectively represented
in a comparison matrix.
10
Example scenario: “A well-established retail business wishes to expand the number of
stores it operates in different states and seeks to know which states will offer the best fit
for its products and services. The task force has determined five major criteria for the
success of the expansion. To determine a weighting for each criterion using a fact-based
method rather than individual perception and emotion, the team uses the analytic hierarchy
process.” Retrieved from [1]
45
Cost of living
Large population
Large airport
Higher average income
Large university
Sum of normalized rankings
Weighting (%)
Analytical Hierarchy Process (AHP) in use
Cost of living
1
1
1/3
1/9
1/5
0.33
7%
Large population
1
1
5
1/3
3
0.93
19%
Large airport
3
1/5
1
1/9
1
0.34
7%
Higher average income
9
3
9
1
9
2.77
55%
Large university
5
1/3
1
1/9
1
0.63
13%
Total
19
5
100%
5 8/15 18 1/3 1 2/3 13 8/15
Table 4: An example of Analytical Hierarchy Process (AHP) in use.
In Table 4, “Higher average income” is 9 times more important than “Large
airport” and as such, gets a relative value of 9, where as “Large airport” gets
9 times less, or 1/9.
When all pairs are compared, the data is used to calculate individual weight
of each criterion. If the Analytical Hierarchical Process is ended at that
point (i.e. not making use of the hierarchical feature of AHP) the ranking
can be calculated (see Table 4 ) and used to prioritize the criteria. However, if
desired, extra levels of sub-criteria can be introduced. For example, imagine
having to decide between several different average income levels (i.e. the subcriteria of “Higher average income”), several different population sizes (i.e.
the sub-criteria of “Large population”) etc. – all being attributes of different
cities. Repeating the process for the sub-criteria (i.e. city attributes) will
produce separate weights (and rankings). Combining the data from the two
levels of criteria can be used to choose the best city (based on the criteria
used).
46
Unfortunately, studies have shown that AHP is not suitable for large number
of requirements [41, 45]. In fact, in the software domain, AHP is commonly
used without considering its signature hierarchical feature, neither with multiple aspects [12] in order to avoid the exponential increase of complexity.
Doing so practically turns the method into the simple “pair-wise comparison
technique” [34, 35, 42].
AHP is potent and popular, however its high complexity renders it unsuitable as a main requirements prioritization technique for requirements-heavy
software development projects.
4.3.3
Cumulative Voting
Cumulative Voting (aka 100-Dollar Test, CV) is a simple, straightforward
prioritization technique better know for use in fields such as political elections and company board elections [15, 12]. Cumulative voting is essentially
a voting system that requires each relevant stakeholder to distribute 100 fictitious dollars (money, points, etc.) across the objects that needs prioritizing.
The stakeholders distribute all 100 points as they see fit, giving more points
to object they feel are more important. In requirements prioritization, the
points assigned to a requirement reflect its relative importance in relation to
the rest of the requirements.
Not nearly as popular as AHP [51], reports show that Cumulative Voting’s
popularity has grown in the past years (in requirements prioritization [62]
and prioritization of process improvements).
One inherent problem of CV might be particularly relevant to game development. As we established earlier (see 4.2.3 Scrum), agile is a favorite
development methodology in the game development business. In the case of
requirements change (addition, removal and/or editing of requirements) - a
common occurrence in agile – requirements prioritization is applied again.
CV’s weakness shows up once the same stakeholders apply it again on the
same list of requirements, since they can bias their evaluation the second time
around if they did not get their favorite requirement as top priority. What
could happen the second time around is that stakeholders might spend all
their points on a single requirement hoping to bump its relative priority.
One way to fight this is by limiting the maximum spendable amount on
47
a single requirement [40], and by doing so introducing the problem where
stakeholders are forced to not prioritize as they see fit.
4.3.4
Numerical Assignment
Ranking in Numerical Assignment (NA) is based on an ordinal scale, as
opposed to weighting requirements on a ratio scale. The result is that requirements in the same group do not get unique priorities.
Numerical (also Numeral) Assignment, the most common prioritization technique [33, 16], is a simple approach that places requirements (or other objects) into different priority groups. Usually, requirements are split in three
priority groups [40, 75] that stakeholders can relate to, such as critical, standard and optional. Ambiguous terms such as high, medium, low [86] will
produce unreliable results since stakeholders might view them differently.
An inherent problem of NA is that stakeholder tend to view most requirements as critical [39, 75], with experiments showing that 85% requirements
end up categorized as critical, 10% standard and the rest as optional [86].
4.3.5
Ranking
Prioritization techniques based on the ordinal scale are typically simpler
than those weighting requirement importance, but none get simpler than the
ranking technique. In contrast to numerical assignment, ranked requirements
hold no ties in rank, thus the most important requirements is ranked 1 and
least important n (for n requirements) [12]. Even thought requirements hold
unique ranks, it is impossible to tell the relative difference in in importance
(possible in CV and AHP). There is no single ranking method, but popular
ones include binary search tree and bubble sort algorithms [34].
In the context of video game development, the ranking techniques do not
work so well, unless there is a single stakeholder responsible for ranking
requirements, as it might be difficult to align several views (e.g. by taking
the mean priority) and could result in ranking ties among requirements in
the list [12].
48
4.3.6
Top 10 Requirements
By far one of the simplest techniques is the Top 10 Requirements technique.
Instructions on how to use this technique can be found in its name - it is
that simple.
Presented with the list of requirements, stakeholders pick their favorite ten
requirements and put them in an unordered list. Lauesen [39] suggests this
method as particularly suitable for multiple stakeholders of equal importance. He adds that taking the average of all top-tens (i.e. from all stakeholders) is not a good idea as it might lead to some stakeholders not getting
any of their top requirements in. Alternatively, essential requirements need
to be defined beforehand. Unfortunately this could bring another set of problems and dissatisfy all potential users as opposed to completely satisfying a
portion of the customers.
4.4
Creative element support in requirements prioritization
For the purpose of designing a requirements prioritization/ordering technique, we have set out to define a specific metric that stimulates creativity
in the development process. Such a technique should ideally help focus work
on the core (creative) vision of the project first, and do so by estimating how
dependent the core vision is on each requirement.
Earlier we talked about the creative element of video games in the context of
the development process and concluded that: the creative element in entertainment software is a multifaceted concept that intertwines the developer’s
personal capacity to produce novel work and recognize its appropriateness
with his/her interpersonal abilities to utilize domain knowledge, values, tools
and best practices. Ideally, a tailored prioritization technique for the gaming
industry should strive to promote and foster every possible aspect the creative element, and where it cannot succeed to do so, it must not degrade the
creativitive process or introduce any significant overhead and/or complexion
to the game development process.
49
(3)
*
Creative
requirements
prioritization
te
* requirements
prioritization
n iq
le m ent effects
on
(2)
ue
s
iv
Creat
up
ee
po
RP
tiv
ts
Cr
ch
rt i
*
n RP
Cre
ati
ve
De
nit
ion
*
n RP
ts
up
po
rt i
en
te
r
nt in entertainm
me
en
ele
t
ea
*
e
en
RP
iv
Creat
m
ele
Creativ
ity i
n
(1)
ftware
t so
en
inm
ta
are
ftw
so
ity
ativ
cre
of
ee
le m
en
* requirements
prioritization
Figure 13: Outermost layer of the research scope diagram: creative element
support in requirements prioritization.
After reviewing some of the most referred prioritization techniques in academic literature we discovered they hold no major inhibitors to the game
development process nor do they clash with the creative element. Regardless of their popularity, simple requirements techniques such as Numerical
Assignment, Ranking and Top 10 fall short of delivering comprehensive prioritization. On the other hand, capable techniques such as Analytical Hierarchy Process are still avoided due to their high complexity when dealing
with a large requirements list.
50
The following table summarizes and compares the key characteristics of the
reviewed five popular requirements techniques [12].
Technique
Scale
Granularity
Sophistication
AHP
Ratio
Fine
Very Complex
CV
Ratio
Fine
Complex
Ranking
Ordinal
Medium
Easy
Numerical Assign.
Ordinal
Coarse
Very Easy
Top-ten
-
Extremely Coarse
Extremely Easy
Table 5: Comparison of popular requirements prioritization techniques based
on measurement scale, granularity of analysis, and level of sophistication of
the technique [12]
Looking at Table 5, one can easily conclude there is no single best or worse
technique. As established earlier, software developers frown upon techniques
with higher level of sophistication as they end up confusing stakeholders
and/or get too complicated when used to prioritize a large amount of requirements. However, all those tools are good in certain situation and if
used wisely can prove very helpful. Moreover, there are some successful
derivatives based on these techniques such as Planning Game [11], which is
a marriage of Numerical Assignment and Ranking, combining their weaknesses and overcoming some of their inherent weaknesses. Nonetheless, none
of these techniques offers support for the creative element in video game
development.
51
Creative element effects on requirements prioritization
RP
*t
e
ch
ni
qu
tiv
ee
le m ent effects
*
on
R
(2)
es
rt i
Cr
ea
(2)
n RP
R
iv
C re a t
po
on
up
Cre
ati
ve
(3)
Creative
requirements
prioritization
P*
le m ent effects
en
te
rt
ts
ee
(1)
n
nt in e tertainm
en
me
t
ele
P*
tiv
Creativ
ity i
n
are
oftw
nt s
me
ain
Cr
ea
vity
ati
cre
of
are
ftw
so
Defi
nit
ion
4.5
ee
le m
en
* requirements
prioritization
Figure 14: Inner layer of the research scope diagram: creative element effects
on requirements prioritization.
In order to discuss the effects of the creative element (see 4.1.4 Creative element in entertainment software) on requirements prioritization suitable for
game development, we need to deconstruct the term into its building blocks
and see if and how they might be affected. Our definition of the creative
element uses Csikszentmihalyi’s Systems Model of Creativity (see 4.1.3 Csikszentmihalyi’s Systems Model of Creativity) as the basis for understanding
the term and thus shares the model’s three-component structure.
[Developer’s] personal capacity to produce novel work - the requirements prioritization technique should be limited to methods that do not
impair developer’s capacity to produce novel work in any way. Ideally, the
method should stimulate this capacity;
[Developer’s] ability to recognize the [game’s] appropriateness - the
requirements prioritization technique should be limited to methods that do
not hamper developer’s ability to recognize the game’s appropriateness and
at best - enable it; and finally
52
[Developer’s] interpersonal abilities to utilize domain knowledge,
values, tools and best practices – the requirements technique should be
limited to methods that enable developer’s interpersonal abilities to utilize
domain knowledge, value, tools and best practices.
Creativity’s effects on requirements prioritization does not simply limit the
choice of requirements prioritization techniques – it necessitates the development of a domain-specific solution. This is not to say that game development
cannot benefit from already existing requirement prioritization techniques,
on the contrary. Available techniques are perfectly adequate for the universe
they inhabit, however the universe of video game development expands regular software’s boundaries by introducing another dimension immeasurable
by classical solutions. This dimension is creativity - the artistic aspect of
entertainment software.
In order measure this extra dimension, it is clear that a brand new solution
needs to be developed - a prioritization solution that focuses solely on the
artistic aspect of video games. Classical methods for requirements prioritization could still be used, but only after the initial intelligent ordering with
a solution that takes into account creativity. Such a solution should be build
using guidelines based on literature and industry insight. Distilling the creative element effects on RP, we defined the following design guidelines for
developing the CRP solution:
Literature guidelines
CG.1 stimulate developer’s creative capacity
CG.2 enable developer’s ability to recognize the game’s appropriateness
CG.3 enable developer’s interpersonal abilities to utilize domain knowledge, values, tools and best practices
Given all of the above, we set off to complete the design criteria for a domainspecific solution and ultimately develop it. The following chapter looks to
the real world in order to complete the design criteria for the development
of the Creative Requirements Prioritization solution.
53
5
Empirical findings
This section summarizes the author’s efforts to investigate the gaming industry’s perspective on requirements management and more specifically - on
requirements prioritization and the corresponding ideas, notions and concepts proposed in our research.
Following the emergent design flexibility strategy (see 3.1.1 Design strategy) by keeping open to adapting the inquiry as understanding deepened
and situations changed, we successfully managed to elicit and outline various perspectives and lines of thought that paint a fundamental picture of
the industry’s take on the notions and hypotheses presented in our research.
This was achieved by means of semi-structured interviews that introduced
the respondents to ideas and question areas of the research and loosely established the tangent and direction of conversation. Ideally, future research
should further investigate the issues reflected here, transcending the scope
and challenging the presented notions in the respective context.
In order to preserve empathic neutrality and mindfulness (see 3.1.2 Data
collection and fieldwork strategy) we corresponded with 11 industry experts
(including programmers, designers and independent developers) with diverse
responsibilities and expertise. Among the respondents are employees of large
and small game studios as well as independent game developers who agreed
to collaborate by providing precious insight and/or by helping design the
central product of this research – the Creative Requirements Prioritization
model (see 6. Creative Requirements Prioritization).
The following sub-section discusses all research-relevant industry expert views
and lines of thought expressed in response to the ideas of Creative Requirements Prioritization.
5.1
Approaching the data
Due to the highly explorative nature of our research, interviews and surveys
were used solely as a means to introduce respondents to the research area
and conceptual ideas (i.e. the introduction of the creative vision as a dimension/aspect in requirements prioritization) and catalize a conversation in the
specific direction.
54
“It wasn’t curiosity that killed the cat.
It was trying to make sense of all the data curiosity generated.”
– Halcolm11
After codifying all 13 correspondence threads, it was time to cut down the
data set to a more manageable size before any analysis on it was done. This
was achieved by stripping away the irrelevant to the research topic parts of
the conversations – a considerable amount of text that glued the researchimportant parts together. The relevant data subset is comprised of any
comment; statement or idea expressed by the respondents in the context of
the subjects discussed in this research.
When it comes to qualitative data analysis, there is a horde of methods to
approach it [55], none of which are precisely defined. The reason behind
this is that each qualitative research and the empirical data collected is
unique to the specific research. Even though suggestions and guidelines
are available, it is the researcher’s responsibility to find and/or devise the
most appropriate methodology fitted to the case at hand in order to analyze
the data as effectively as possible. No precise or agreed-on terms describe
varieties and processes of qualitative analysis [55].
In the spirit of scientific rigor, the qualitative data analysis needed to be
approached in a systematic way, following a proven strategy. Seidel [71] describes the framework of qualitative data analysis as the process of noticing,
collecting, and thinking about interesting things (see Figure 15) – a fundamental model that allows the researcher to move in different directions
according to the data and the context of the research. While there is a great
diversity of QDA practices, Seidel [71] argues that all forms of QDA are
based on these tree “notes”.
11
Halcolm made his debut in the first edition of Patton’s book [55] in 1980 as a qualitative inquiry muse and Sufi-Zen teaching master who offered stories that probed the deeper
philosophical underpinnings of how we come to know what we know–or think we know.
Halcolm’s musings, like his name (pronounced slowly), lead us to ponder "how come?"
55
2
Qualitative Data Analysis
Figure 1. The Data Analysis Process
As Figure 1 suggests, the QDA process is not linear. When you do QDA you do not simply Notice,
Figure
Theabout
Data
Analysis
Process
to Seidel
[71]
Collect,
and 15:
then Think
things,
and then write
a report. according
Rather, the process
has the following
characteristics:
C
Iterative and Progressive: The process is iterative and progressive because it is a
cycle
keeps repeating.
For example, when
you are thinking
about things
Agar [71] suggests
a that
method
of intensively
examining
a small
bityou
of data
also start noticing new things in the data. You then collect and think about these
rather than intensively
coding
data
– synthesizing
it as “a little bit of data,
new things.
In principle
the process
is an infinite spiral.
is recursive
because
one part can call
you backthe
to a noticing,
and a lot ofC rightRecursive:
brain” The
- aprocess
method
that
recursively
follows
previous part. For example, while you are busy collecting things you might
simultaneously
start noticing
new things toby
collect.
collecting and thinking
steps
as described
Seidel [23, 71]. This trialC
Holographic: The process is holographic in that each step in the process contains
and-error methodtheworks
best when
working
a manageable
dataset as it
entire process.
For example,
when youwith
first notice
things you are already
mentally collecting and thinking about those things.
focuses researcher’s
efforts on understanding respondents’ position as best as
Thus, while there is a simple foundation to QDA, the process of doing qualitative data analysis is
possible
and the reasoning behind it, which is quite labor-intensive. Employcomplex. The key is to root yourself in this foundation and the rest will flow from this foundation.
ing Agar’s method over codifying-centered QDA alternatives was the logical
Noticing, Collecting, Thinking about Things
choice as our research-relevant dataset is on one hand not exceedingly voIn the next sections I further elaborate on the notice, collect, think process. The primary vehicle is
luminous
and ofon
thea jigsaw
otherpuzzle.
– respondents
exhibited
views that
the analogy
solving
Then I show some
of the ways invery
which diverse
the notice, collect,
process
been expressed
in the writings
of qualitative
scientists. Finally,
I explore
hardlythink
relate
tohaseach
other and
thus we
found social
it necessary
to look
into each
two alternative analogies. One is the “multi threaded DNA” analogy (Agar, 1991). The other is a
map analogy
on the
of “topographical”
maps and
“ad then
hoc” maps.
separate
majorbased
train
ofideas
thought
first, and
only
contrast it against the
other.
5.1.1
From questionnaire to conversation
The initial set of questions was based solely on literature and logical assumptions. Understandably, the questions evolved after the initial interview with
the first respondents. Naturalistic inquiry [55] and Emergent design flexibility strategy (see 3.1.1 Design strategy) guides into this explorative process,
as understanding of the problems observed deepens.
The questions presented in this section are the ones present in the final
version of the questionnaire and interview protocol.
56
1. Do you have any experience with Requirements Prioritization methods/algorithms in the preproduction (i.e. vision or design) phase of
game development? Please elaborate on what technique(s) you have
used or encountered. Note: If you don’t know the name of a technique
you have used or it doesn’t have one, explain its steps briefly. - This
question explores the respondent’s general knowledge in the field of
requirements prioritization.
2. Do you follow the trends in game design? If yes, how? If not, why not?
Note: e.g. publications, magazines, journals, books, forums, special
interest groups etc. - Answering this question, the respondent should
show if he has interestsen using the latest solutions.
3. Have you read anything on requirements prioritization? If yes, what
and where? Note: e.g. publications, magazines, journals, books, forums, special interest groups etc. - Checking if the respondnet has any
specific interest and/or knowledge in requirements prioritization.
4. Do you think video game requirements differ from regular software (e.g.
spreadsheet software) requirements? If yes, how? If not, why not?
Please elaborate. - The first very important question with relation to
the research at hand. The answer could be indicative to the validity
of the hypothesis of this research.
5. Whose responsibility should requirements prioritization be? Note: e.g.
designers, developers, the person that came up with the idea, the target
group, other stakeholders, all of them etc. - The answer could provide valuable insight with regards to the stakeholders involved in the
proposed by this research’s domain-specific requirements prioritization
solution.
6. Would you say that video game requirements have a creative / entertainment value? If yes, how would you define that value? If no, why?
Please elaborate. Note: By creative/entertainment value of a requirement, we refer to the degree to which the project’s core vision relies on
this particular requirement. - A pivotal question testing the hypothesis
that some requirements add more value to the core vision of a game
development project than others.
57
(a) How would you measure this creative value? Please give example and/or elaborate. - This question encourages the respondent
to use his/her understanding of the domain and the problem to
propose a solution.
(b) Who would measure this creative value (e.g. designers, developers, other stakeholders, all of them etc.)? Please elaborate. - In
the case where respondents believe that some requirements add
more value to the core vision of a game development project than
others, this question can provide valuable knowledge/ideas for the
solution this research aims at developing.
7. If you have any ideas or suggestions on Requirements Prioritization
for game development that have not been expressed in your answers
until now, please express them now. - Similar to the ‘Other’ category
of fixed answers with closed question surveys, this question allows for
total freedom of expression of opinion on the issues presented with the
previous questions. It aims at collecting any ideas that the respondent
might have missed to share during the interview/questionnaire.
The questions above appeared as they were presented in the questionnaire
sans the background knowledge provided for respondents unacquainted with
the terms present in the questions.
The truth of the matter is that questionnaires provided more questions than
answers. Despite introduction into the research topic, many respondents felt
lost in this open-question questionnaire as the topic is quite unique and (as
some admitted) this discouraged them from providing deep answers. After
discussing this issue, respondents who showed interest in the topic felt this
issue should be tackled in the form of free conversation, as pre-structuring
an unexplored area of research seemed inadequate.
In the spirit of Naturalistic inquiry [55] and Emergent design flexibility strategy (see 3.1.1 Design strategy), from a well defined questionnaire the process
of empirical data collection morphed into 13 conversation threads, each steering the topic in the direction of respondents field of expertise, interest and
comfort. This provided the empirical data bulk needed to fuel our research.
The next section presents the collection of the most interesting to the research at hand views and notions collected from our dataset and discusses
58
them before finally drawing a conclusion to be used as part of the design
criteria for the Creative Requirements Prioritization model.
5.2
Key expert notions and attitudes
There were eleven main views that were discussed in the conversations with
our respondents. These views paint a simple but colorful (and sometimes
conflicting) picture of certain research-relevant industry insight. All thirteen respondents discussed all eleven notions, although some respondents
felt more comfortable expressing their attitude towards certain notions than
others. This is easily explained by the difference in field of expertise and work
environment of respondents. For example, single independent game developers could hardly comment on organizational structure implications to game
development. The following table shows the positions of all 13 respondents
(anonymously):
Table of respondents
Codename
Nationality
R1
BG
R2
UK
R3
BG
R4
US
R5
US
R6
RU
R7
NL
R8
BE
R9
US
R10
US
R11
PT
R12
BG
R13
UK
Role
Programmer
Programmer
Lead Programmer
Software engineer
Designer
Programmer / Independent game developer
Story writer / Script writer
Programmer / Independent game developer
Designer / Independent game developer
Independent writer / Designer
Producer
Visual artist
Independent game developer
'
Table 6: Table of respondents
The eleven notions and attitudes towards them were easily identified based
on both their relevance to the research scope and enthusiastic attitude of
the respondents discussing them. These are discussed one by one in this
subsection.
59
5.2.1
Creativity and "enjoyment" are fundamentally not quantifiable
Since we are trying to come up with a way of prioritizing requirements specifically tailored for creative and entertainment software development it was
logical to see what are the industry’s views on aspects such as quantifying, measuring and/or predicting the enjoyment value of software products.
Suffice to say neither designers nor programmers knew of any existing mechanisms to do so and even went as far as to declare it impossible.
“Creativity and ‘enjoyment’ are fundamentally not quantifiable. If we could attach numbers to these things, rest assured that someone would have invented the analogue of an
actuarial position for rating these subjective qualities, and there would already be a thriving industry that is extremely good at this.” – R4
This attitude was not surprising to us as there was no sign of such mechanisms for measuring these subjective aspects in academic literature either.
At this point of our research, the romantic idea of prioritizing software requirements by how enjoyable or entertaining a feature might be had fully
vanished. On the other hand, the idea of preservation of creative vision
during development began to look like the right direction for a tailored requirements prioritization solution for the industry.
5.2.2
Process implies rigidity
The proposition of introducing new processes in the industry is generally
frowned upon due to several reasons. One particular reason is the believe
that process equals rigidity. Academia is happy to design new solutions such
as the one we are building here, however there is the constant fear of loosing
flexibility.
“There are two kinds of software companies. One kind is rigidly attached to process, often
informed by academic research, and spends tremendous amounts of money and time on
things like certifications, process compliance, and so on. The other type could be seen as
chaotic, ad-hock, but also flexible.” – R2
The formulation of this idea is a bit extreme as it paints a black-and-white
60
picture. As the famous cliché states - the truth is somewhere in the middle.
There are many development solutions designed with flexibility in mind that
successfully aid and promote flexible software development environment (e.g.
SCRUM). This is not to say this is not a valid concern – on the contrary,
it is – and we take it very seriously. Any solution born from this research
takes into account the need for flexibility, easy adoption and introduction of
no substantial overhead.
5.2.3
Process kills creativity
Previously we mentioned the idea of working towards a requirements prioritization solution that promotes the preservation of the creative vision of the
development project at hand. This evolution of the idea created some waves
among the respondents. The main concern now was not that process would
introduce rigidity, but that process kills creativity.
“Being strangled by process and procedure is a surefire way to kill off creative drive and
innovation.” – R13
That would suggest that the idea of a process that promotes the creative
vision is fundamentally flawed. The first opinions on the idea were all from
programmers and largely agreed that process tends to suppress creativity.
However designer respondents had a contrasting view that suggested knowledge and usage of theories and methods provide perspectives that can ultimately help solve a problem and as any tool need to be used correctly.
“I regularly see writers saying they are afraid of reading books with theories of how to
write good fiction, because they are afraid this will somehow decrease their creativity.
Personally I think this fear is absurd. It’s the same for decision-making methods. Being
familiar with a theory or a method doesn’t mean you have to use it slavishly. You don’t
somehow give up your own ability to make judgment calls just because you study someone
else’s strategy for making such judgments. Being familiar with others’ theories is just like
asking yourself, "Ok what would Joe do in this situation?" to come up with a second and
third opinion. Looking at a situation from more than one angle helps one make balanced
decisions that won’t turn out to have missed an important factor.” – R7
At this stage of exploration it became clear that if an RP solution promoting
the preservation of creative vision was to be created, it would have to be
61
designed with simplicity and freedom of application in mind – i.e. giving the
users the freedom to use it when/if needed.
5.2.4
Process is just a shield that shifts responsibility
On the topic of process, one more concern was noted – namely that process
is just a shield for shifting responsibility.
“Process is the shield which management uses to deflect blame away from themselves
because they have realized the most important secret in software development: project
failure is due to poor project management.” – R3
This might be the case with rigid processes that leave no freedom of action to
its users, however loosely defined processes that aim at promoting healthy
practices and the fair distribution of responsibility such as SCRUM have
proven to be quite successful and widely adopted in the industry. An effective
process needs to either not impair project management or work towards
improving it. In conclusion, shifting responsibility is not an attribute of
processes by definition and they should not be designed and used as such.
5.2.5
Profitability vs. risk is the only important metric
Looking for the most important metrics when prioritizing requirements in
the gaming industry we found that there are no traces of prioritizing using
creative vision in any form. Especially in the initial stages of the design and
implementation, the only important metric seems to be profitability vs. risk
which can be broken down to the traditional three prioritization dimensions
value, cost and risk.
“If it seems like a profitable idea, the product enters full design, reviewed periodically for
profitability vs. risk, hopefully not cancelled, and goes through the rest of production and
post-production and marketing and eventual sales cycle, all based on profitability.” – R11
“A game designer may present a design to an executive or group of executives, but there
is pretty much 0% chance that the final produced game will look identical to the design.
And the first batch of changes is going to be an evaluation of whether anything in the
design is going to be more expensive than it’s worth to implement.” – R5
62
Estimating potential profitability and risk for RP is essential for any software
development project, not only for the gaming industry and there is a plethora
of methods (see 4.3 Overview of requirements prioritization) that use those
three aspects in order to do it. However, if video-game software is considered
something more than just regular software and is viewed as a creative effort
with its artistic values and similarities to film art – it is logical to also take
the project’s creative vision into account when deciding on which features to
implement.
5.2.6
Creative vision works only for art
Steering the conversation into the controversial direction of creativity, creative vision, art and how it all relates to video-games revealed a clear duality
in opinions from respondents. The initial several conversations on these topics were discussed with programmers and their opinion painted a somewhat
bleak picture for the video-games-as-art-form hypothesis:
“Creative vision is fine if you’re making a painting or a sculpture. When you are making
a product, it is peripheral at best.” – R1
with some views being as negative as:
“Nowhere in the corporate process of making games is the creative or entertainment value
a consideration. During production some managers will use it to help prioritize the lesser
object creation order, but again that is a minor side consideration fairly late in the cycle,
and only used as a minor detail” – R4
In contrast, the respondents with roles more involved in the artistic aspects
of game development (i.e. designers, writers, artists etc.) defended the idea
that video-games have become an expressive medium with modern society.
However, most agree that:
“Making a movie is really a better comparison to the process of making a game than
making productivity software is” – R9
63
Regardless of the controversy behind the debate over whether games are
a form of art or not, popular culture has clearly embraced the idea already. For example, the recent exhibition12 at the Smithsonian Art Museum in Washington DC explored the 40 year evolution of video-games as an
artistic medium. Even research argues that “by any major definition of art
many modern video-games should be considered art” [74][27]. In her 2009
TEDxUSC talk13 , indie game developer and producer Kellee Santiago (Flow
and Flower and other titles) championed the idea that video games should
be viewed as art and should be developed accordingly. In her interview14
at the Smithsonian Art Museum exhibition, Santiago expressed her hope for
an industry-wide change in this direction and approach video games as a
creative medium rather than a product.
Santiago, among other game developers has proven that it is very much possible to produce successful and highly enjoyable titles viewing the process
and resulting game as an art form rather than a product. Therefore, an instrument that takes into account all aspects of this emerging paradigm would
potentially enhance and support the development process thus ultimately resulting in a finer game. Ultimately, the Creative Requirements Prioritization
solution aims at serving as one such instrument, built specifically to support
successful game development process, whilst preserving creative vision and
staying out of the way of artistic expression.
5.2.7
Creativity is a group activity
While there was a clear separation between respondent’s opinions on the
issue of video games perceived as artistic expressions, all respondents agreed
video games are a creative effort and more importantly - a group effort.
12
The Smithsonian American Art Museum invited the public to help select the video
games to be included in the exhibition. The 240 games on the ballot were selected by
Chris Melissinos, who worked with the museum and an advisory group consisting of game
developers, designers, industry pioneers, and journalists. The games were selected based
on a variety of criteria, including visual effects, creative use of new technologies, and how
the game fit into the narrative of the exhibition. Voting took place between February
14 and April 17, 2011. More than 3.7 million votes were cast by 119,000 people in 175
countries!
13
“Kellee Santiago:
Are Video Games Art?”
seen on 10/Feb/2013 at
http://stevens.usc.edu/Kellee_Santiago__Are_Video_Games_Art_.flv
14
"The Art of Video Games: Interview with Kellee Santiago, Jenova Chen, and Robin
Hunicke" seen on 10/Feb/2013 at http://www.youtube.com/watch?v=OG2dO9E2WHE
64
“Creativity and systematic evaluation of ideas have to work hand in hand to produce any
finished, salable product. Evaluation and editing, done right, improve creativity instead
of stifling it.” – R11
Undoubtedly, some of the most important issues that might arise while group
creativity takes place are the inevitable conflicting ideas and alternative interpretations of the creative vision. As such, an environment that enables
group creativity needs to allow for collaborative work and constant communication between stakeholders involved in the process of building and
implementing the creative vision of a project.
“A game design is more or less a piece of art, and as such it can only arise from the
personal preferences of the game designer. My designs first and foremost exist to please
me. But the designer is only one person, if they have a few preferences, which are way out
of line with the target demographic it’s completely appropriate for these to get overridden
during the development process even though that will probably annoy the designer. Being
an artist is like that, trying to turn art into a commercial product always requires a bit
of hackwork and pandering.” – R5
Constant group supervision and collaboration between stakeholders on the
creative vision throughout the project’s life will help establish a shared vision
and focus the group effort in a single, agreed upon direction. Having in mind
all this, the CRP solution needs to enable and support group creativity with
its peculiarities.
5.2.8
The gaming industry is a hot bed of imitation
While investigating the possibility of prospective features evaluation during
early stages of the game development process, one specific aspect of the industry came into light – imitation. For example, after Blizzard’s World of
Warcraft, a wave of game studios hoping to imitate the success, developed
uncomfortably similar games such as - Taiwanese developer Runewaker Entertainment’s Runes of Magic; or Gameloft’s “tribute” to Blizzard - Order &
Chaos Online, which is practically WoW for mobile devices.
“The biggest problem with trying to measure a discrete aspect of a game and giving it
a numerical ranking is that games are rarely just the sum of their parts. Games are
65
cohesive systems; all interacting. When evaluating the relative worth of a single piece it
must be play-tested through the whole, but there in lies the problem - the "whole" is itself
incomplete and so the evaluation is at best a cursory value.
So how do people get around this? Simple - they copy. They copy features from other
games which are complete and thus can appreciate the nuances of the feature both individually and collectively. Yes the games industry is a hot bed of "imitation" but so are
most human endeavors I would argue. The bigger the game the less "innovation" you will
see, since the risk of "messing up the formula" is very high and they are very risk adverse.”
– R3
As the respondent above states, evaluating the relative worth of a single piece
(i.e. a requirement) is not possible since its value cannot calculated unless
the game is in a complete and/or playable form. This is why CRP needs
to be able to evaluate candidate ideas (or requirements) from the get-go,
independent from other requirements. This is possible, because our solution
will do its job by referring requirements to the core creative vision of the
project, and not other requirements.
Furthermore, Creative Requirements Prioritization should allow for projects
that aim at imitating another product, regardless of the unoriginality. Imitation - be it a remake, a sequel or a blunt copy of somebody’s idea – should
not be an issue with CRP, as the solution only requires a clear definition of
the core creative vision of the project, as unoriginal as it may be. However,
it seems that CRP would benefit creative and original projects more since
besides prioritizing requirements, it should also help preserve the core creative vision and communicate it between stakeholders in order to deliver a
product that is closer to the expectations.
5.2.9
Introduction of novel ideas requires "champions"
During game development, it is common for new features to emerge as someone comes up with “this great idea!” and it needs to be introduced into the
project and communicated and evaluated by the relevant stakeholders.
“I think the way it works now is it takes a "champion" of an idea, be that a designer,
programmer or producer (they are the usual ones I see championing ideas). Maybe they
came up with the idea from another industry or some stroke of insight or random chance,
but for whatever reason they really like it. People will always resist new ideas, that’s just
human nature.” – R11
66
The so-called “champion” of the idea has the difficult task of convincing the
relevant stakeholders that it is worth investing resources into. This task can
be made easier if the idea is obviously in tune with the core creative vision
of the project. CRP should potentially make this process faster and easier
by providing a clear definition of the core creative vision of the project. If
this new idea is resonating with the core creative vision, CRP should then
help communicate the prospective added value by this new idea (i.e. new
requirement) to the relevant stakeholders. Finally, if the idea is within the
scope of the project, developers can proceed to its implementation.
Respondents all agreed that if a core creative vision was to be used as a
means of evaluating requirements, it needs to be defined simply and clearly
so that it is unambiguous. Furthermore, it has to be communicated after
any change to it and has to be available to all stakeholders of the project at
hand. CRP should satisfy all these conditions without adding any significant
overhead to the development process.
5.2.10
Feature creep is the result of the absence of a shared vision
When talking about the emergence of unscheduled features during development, one cannot overlook the popular issue of feature creep – features that
don’t fit in the boundaries of the predefined scope of the project. Respondents agree that in the gaming industry, feature creeps are a major problem that often leads to scheduling problems and could potentially destroy a
project with a limited budget.
“Companies which can afford to (ie Rockstar, Blizzard, Sony, Valve etc..), invest the time
to make it fun and are not slave to the schedule. Those who can’t press on and hope for
the best. What can be done about this? I think the reason why things don’t turn out to
be "fun" is because lack of shared vision and creative leadership. People usually have an
idea of what is "fun", but for something to be "fun" a lot of factors have to align, and
without a shared vision of why or what is fun in a game, people can very often put in less
than satisfactory effort in the needed aspects to make it so.” – R10
Discussing the issue of feature creeps with respondents and the possible
remedy led to the consensus that the best way to avoid this schedule- and
resource-destroying malady is to have a shared vision. A shared vision should
67
help stakeholders take better actions when it comes to adding features. The
Creative Requirements Prioritization solution must ensure the core creative
vision of project is shared across all stakeholders as well as help sift out the
vision-relative features from the rest.
Furthermore, respondents pointed at the occasional need to cut down features in order to bring a game closer to the initial vision, as sometimes
addition of new features strays the product away from it.
“Cutting features? I think this is needed actually. Back to the previous, point games are
a collection of systems, like the reverse analog of "feature creep", sometimes what it takes
to make a game fun is to remove a feature. That’s just the reality, after play through,
after some insight, maybe you don’t need a super smart horse AI? Maybe. Even that
being said, cutting the wrong feature for the wrong reasons can equally kill a game and
I’ve seen that happen too. So how do u know what to cut? Again goes back to above you
need to have a shared vision and creative leadership on what makes the game fun.” – R8
The need for a controlled and communicated creative vision of the project is
obvious. This is why CRP could improve the development process. Not only
will it take care of defining a core creative vision of the project, but it will
help communicate it and help distinguish vision-related ideas from feature
creeps.
5.2.11
Organization culture determines creativity flow
Organizational structure within gaming companies was brought up during
conversations with respondents as an important factor when discussing communication of a project’s shared creative vision.
“I’ve noticed that each company has their own "way" of doing things. Some are very much
are hierarchical and ideas only flow from the top down, no exceptions. Others are more
egalitarianism where ideas can come from the lowest person on the ladder (a temporary
tester for instance) and are seriously considered and evaluated without prejudice. The
thing is both systems work, if you have a core of highly competent and creative individuals
they really don’t need external "assistance", or if you have an open environment where
all share in the creation of a product this too can work.” – R11
Designing the CRP solution needs to take into account these two different
environments and make sure it works equally well in both conditions. For
68
instance, regardless of the organizational structure, the core creative vision
needs to be communicated between all relevant stakeholders and if changes
are made to it during the entire life of the project.
Whatever the organizational structure in a company, there was a consensus
among respondents that if a utility such as CRP was to be employed, it would
need to be simple and require negligible effort to understand, implement and
use. The reason being that stakeholders involved in designing, developing
and delivering the game to the market all need to share the same vision of the
game, which might mean that people (possibly from different departments)
with different backgrounds need to be able to use CRP’s method of defining
and building the core creative vision.
5.3
Industry expert guidelines for CRP
In the end, the conversation with industry experts provided a set of guidelines
that comprise the most substantial part of the design criteria used for the
development of a creative requirements prioritization technique. Initially, all
respondents were confronted with the idea of pivoting major development
decisions around the creative vision of a project – an idea based on the notion
that video games could be viewed as art and their development can be carried
out accordingly. An idea that (not surprisingly) immediately resonated with
the more creative contingent of the respondents – game designers and writers.
Regardless of the obvious division between respondent’s views on what games
are, all respondents agreed that a shared creative vision during development
would most certainly aid a successful project.
Moreover, it was agreed that such a shared creative vision could be used
as a reference point for discerning elements (i.e. requirements/ideas) that
contribute to the project and such elements that are of little importance and
would just burden development. In other words, a well defined and shared
vision would allow for a lean game development, where requirements are not
prioritized solely using the classical dimensions of cost, value and risk. The
introduction of an extra dimension such as the shared creative vision would
enhance the prioritization of requirements by sifting key from trivial ideas
and focus the work on realizing the core creative vision15 of the project 15
For example, a simple game like Angry Bird’s core creative vision would be “simple,
intuitive and fun sling game”.
69
the essential most definitive creative vision of a game development
project.
To conclude, we digest the eleven notions and attitudes expressed by our
respondents into guidelines that will heavily influence the design criteria for
the development of the Creative Requirements Prioritization technique.
None of the following guidelines for developing our solution is more important
than any other guideline in the same list – all of these need to be satisfied
when developing the Creative Requirements Prioritization technique.
Industry guidelines
IG.1 Preserve the core creative vision throughout development
IG.2 Keep it flexible and introduce no substantial overhead to the development process
IG.3 Keep it simple and nonintrusive – use it when/if needed
IG.4 Share but don’t shift responsibility
IG.5 Take project’s creative vision into account when deciding on features
to implement
IG.6 View the game as an art form and not a product – stay out of the way
of artistic expression
IG.7 Enable and support group creativity through shared creative vision
IG.8 Be useful for original as well as imitation projects
IG.9 Provide a clear definition of the core creative vision
IG.10 Communicate the core creative vision and any changes made to it
during development with all relevant stakeholders
IG.11 Allow all of the above, regardless of the organizational structure
of the gaming company
The following chapter shows the design rational behind the Creative Requirements Prioritization technique. The acquired guidelines presented above are
used to draft the design rationale and to develop the CRP technique.
70
6
Creative Requirements Prioritization
This section of the paper presents the key product of our research – the
Creative Requirements Prioritization solution - and examines the processes
that led to its development.
The CRP technique is an attempt to embrace the notion that video-game
software is fundamentally different from business software. Contemporary
cultural perception of video games has labeled them as means of artistic
expression and even art with its own dedicated museum exhibitions (elaborated in 5.2.6 Creative vision only works for art). With a growing number
of successful modern game titles being developed as artistic projects as opposed to classical software projects it seemed that there is a need for contextually adequate software development model. After thorough research in
literature on existing techniques it became clear that no available software
project management solution of any kind fosters this emerging paradigm in
game development. Furthermore, parts of the industry have acknowledged
the need for industry-global shift towards a more art-centric development of
video games. Lastly, a dozen industry experts (i.e. developers, designers,
programmers, writers, game artists etc.) that kindly helped our project recognize the need for a shared core creative vision during game development – a
central element in our deliverable, the Creative Requirements Prioritization
technique.
Our work pivots around requirements prioritization during game development with a focus on the creative nature of the products developed, motivated by the contextually inadequate solutions offered by classical software
project management. Before delving into the Creative Requirements Prioritization itself, the following subsection will first go through the design
criteria and rationale behind it.
6.1
Design criteria and rationale
The design criteria used to create the Creative Requirements Prioritization
Model is based on several components of this research. Firstly, it was important to set an unambiguous definition of creativity to be used in the context
of the research at hand (see 4.1.1 Definition of creativity). Next, we reviewed
available academic work on creativity in search for a better understanding
71
of the mechanics of the creative process and contextualized the findings to
the domain of video game development in order to define and understand
the creative element of video game development (see 4.1.4 Creative element
in entertainment software).
Furthermore, we reviewed academic literature on video game development
in order to identify the most common development processes in the field.
Reviewing literature on requirements prioritization provided a list of the five
most widely used prioritization techniques. Using this knowledge, it was important to understand the role of the creative element of game development
in the context of popular requirements prioritization and game development
practices (see 4.4 Creative element support in requirements prioritization;
and 4.5 Creative element effects on requirements prioritization).
Finally, we conducted in-depth explorative interviews and open-question surveys with a dozen industry experts (programmers, designers and other) to
see if our literature-based conclusions conflict with the world practices of
game developers and to seek advice on how to design a solution.
Using the industry insight and our definition of the creative element in video
game development, a list of design criteria was composed. Figure 16 provides
an overview of this process.
Design Criteria for CRP*
DC
Literature on Creativity
Industry Insight
Literature on RP**
Creative Element
Design Criteria
*
**
Creative Requirements Prioritization technique
Requirements prioritization
Figure 16: Design criteria rationale of the Creative Requirements Prioritization techniqe
72
Design criteria
The list of design criteria is a combination of the gathered knowledge from
two sources of our research – academic literature (i.e. literature on creativity
and literature on requirements prioritization) and empirical findings (i.e.
industry insight from our respondents) – illustrated in Figure 16.
Based on the creative element of game development (from Chapter 4 Theoretical foundations), CRP has to:
CG.1 stimulate developer’s creative capacity
CG.2 enable developer’s ability to recognize the game’s appropriateness
CG.3 enable developer’s interpersonal abilities to utilize domain knowledge, values, tools and best practices
Based on industry insight guidelines (from Chapter 5 Empirical findings),
CRP has to:
EG.1 Preserve the core creative vision throughout development
IG.2 Keep it flexible and introduce no substantial overhead to the development process
IG.3 Keep it simple and nonintrusive – use it when/if needed (similar to the
KISS principle16 )
IG.4 Share but don’t shift responsibility
IG.5 Take project’s creative vision into account when deciding on features
to implement
IG.6 View the game as an art form and not a product – stay out of the way
of artistic expression
IG.7 Enable and support group creativity through shared creative vision
IG.8 Be useful for original as well as imitation projects
16
KISS is an acronym for the design principle articulated by Kelly Johnson, Keep It
Simple, Stupid. Variations include "keep it short and simple", "keep it simple sir", "keep it
simple or be stupid", "keep it simple and stupid", "keep it simple and straightforward" or
"keep it simple and sincere." The principle most likely finds its origins in similar concepts,
such as Occam’s razor, Leonardo da Vinci’s "Simplicity is the ultimate sophistication",
Mies Van Der Rohe’s "Less is more", or Antoine de Saint Exupéry’s "It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to
take away".
73
IG.9 Provide a clear definition of the core creative vision
IG.10 Communicate the core creative vision and any changes made to it
during development with all relevant stakeholders
IG.11 Allow all of the above, regardless of the organizational structure
of the gaming company
The following section presents and explains the model, and elaborates on
how it satisfies the different criteria.
6.2
CRP Model
This subsection of the document presents the designed product of our research – the Creative Requirements Prioritization Model - a requirements
prioritization technique specifically designed to help developers nurture the
core vision of game development projects and introduce an extra dimension
en
te
r
nt in entertainm
me
en
ele
t
*
n RP
(3)
Creative
requirements
prioritization
te
n iq
RP
ee
le m ent effects
on
(2)
ue
s
iv e
Creat
po
tiv
up
*
ch
ts
Cr
RP
ea
rt i
*
Cre
ati
ve
Creative
requirements
prioritization
Creativ
ity i
n
(1)
ftware
t so
en
inm
ta
(3)
ity
ativ
cre
of
are
ftw
so
De
nit
ion
for requirements prioritization.
ele
m
en
* requirements
prioritization
Figure 17: Core of the research scope diagram: final step (3) - desing a
Creative Requirements Prioritization solution
The CRP Model is basically a method for defining and sharing a game development project’s creative vision that can be used as a reference system
for choosing the right requirements to implement in order to best realize this
vision.
74
The CRP Model’s structure resembles the geological structure of our planet
Earth. This metaphorical representation gave the name of the three layers of
the CRP Model: tier I - Inner core; tier II - Outer layer and tier III - Lower
mantle (see Figure 17 ). The three tiers give a layered model representation
of the core creative vision, where the innermost layer (or the Inner core) holds
the most important idea of the game – the most definitive feature. Building
on the Inner core are the Outer core and the Lower mantle, each holding
the second and third most definitive features of the game’s core creative
vision. Theoretically, our model can have more than three layers (or tiers),
however it is not advisable and the reason for this will be discussed in the
following section where the application of CRP is illustrated under the most
industry-wide adopted development technique.
A Model for Creative Requirements
Prioritization (CRP)
Lower mantle
Outer core
tier III
tier II
tier I
Inner core
1-3 sentences
1-3 sentences
1-3 sentences
Figure 18: Creative Requirements Prioritization Model
Figure 18 illustrates the structure of the CRP Model with its three layers.
This is in fact the structure of the core creative vision of a given game
development project in the form of a clear definition of what makes the game
interesting. It is “core” because it lacks any unnecessary details and focuses
solely on what the project stakeholders believe is the most definitive aspect
of the game to be developed. This focus is achieved through limiting the
75
definition to as little detail as possible – just so that it retains the main idea
of the game. The codification of the core creative vision is a group effort that
includes all stakeholders involved in the development project (i.e. designers,
programmers, marketing etc.) in order to achieve cohesion and have a shared
vision of the expected product. Each layer of the model is populated with
a few concepts (1-3 short sentences each layer) starting with the innermost
layer (the Inner core) where the most iconic aspect of the project’s creative
vision is situated. The Outer core houses the second set of aspects that
complement definition and finally, it is advisable to conclude the definition
using the Lower mantle. Notice the absence of precise number of concepts
a layer can hold. This is because games can be very different and original,
and can also be quite complex or simple, thus it is up to game developers
to decide how to define and describe their creative vision. However, it is
imperative to keep it as simple as possible in order for the CRP Model to be
most useful, and this is why each layer has to hold at most 3 short sentences
– a number that our respondent industry experts felt was just the right
amount of limitation, referring to the short game design pitches presented
by game designers to executives in the industry.
When the CRP Model is complete with a clear, short and concise definition
of the game in development, it can be easily shared with all relevant stakeholders and used during development as a means to stay focused. Moreover,
when handling such a short definition, any changes or updates that might
occur to it can easily be communicated with all stakeholders (e.g. using
Google Docs or similar tools).
Once the CRP Model is complete it can be used as an extra dimension when
prioritizing requirements by introducing the core creative vision it holds as
a reference model. The procedure of requirements prioritization using the
CRP model is quite straightforward. All one needs to do is compare the
desired idea/requirement to the model and see which layer it relates to, if
any. Figure 18 illustrates a set of requirements (A, B, C, D and E) where
each requirement relates to a different concept within on of the layers of the
CRP Model. Relative to each other, B is the most important requirement
as it relates to a concept within the Inner core; A and C are the next most
important requirements; D and E are the least important requirements. All
these requirements are important to the defined core creative vision, but
if the circumstances do not allow for the implementation of all of those, it
76
is clear which ones are less important and would harm the final result the
least. It is, of course, possible that some requirements are dependent on
others and as such need to be grouped and compared to the model as a
group. Cases where one requirement relates to concepts in more than one
tiers are illustrated and explained in the next sub-section 6.2.1 Example of
CRP in use under Scrum. In the case that the core creative vision does not
rely on a specific requirement, it is clear that it should be implemented only
after all other requirements that the vision relies on are already implemented.
Prioritizing using CRP (example)
tier III
tier II
tier I
requirement A
concept 1,
concept 2...
concept h;
requirement B
concept h+1,
requirement C
concept h+2...
concept i;
requirement D
concept i+1,
concept i+2...
requirement E
concept j.
Project’s creative vision = { concept 1, 2, ...j }
Figure 19: Prioritizing using the Creative Requirements Prioritization Model
Moreover, the implementation of a major requirement that does not related
to the core creative vision should be discussed by the stakeholders as it might
not provide any added value or even compromise and bloat the final product.
6.2.1
Example of CRP in use under Scrum
It is important to note that CRP is no replacement for other requirements
prioritization techniques. Prioritization aspects/dimensions such as cost,
risk, etc. should be estimated using an appropriate for the job technique.
77
For popular prioritization techniques see 4.3 Overview of requirements prioritization.
This example use of CRP under Scrum was outlined with the help of two
of our respondents with extensive experience in agile game development.
The Scrum development environment was chosen as the context for CRP’s
examples, due to its wide adoption in the gaming industry. However, it
should be noted that it has not been tested in the real world - this part of
the research is left as future work.
Requirements prioritization in Scrum is a duty of the Product Owner, and
as such, he/she would be responsible for building, using and maintaining the
CRP Model.
The Process-Deliverable Diagram in Figure 20 overviews the process from
CRM Model building phase to ordering requirements in the Product Backlog
using the CRM Model.
CRP Model building phase
RELEVANT STAKEHOLDERS GROUP
Identify project-relevant
stakeholders
Stakeholder ID
1..*
define
PROJECT’S CRP MODEL
Organize vision definition
meeting
Product Owner
Tier I vision
Tier II vision
Tier III vison
... Tier n vision
1
1
is ordered
accoring to
1..*
ORDERED PRODUCT BACKLOG
Requirement A
Requirement B
...
Requirement n
Order reqirements
Product Owner
Figure 20: CRP Model building phase illustrated using a Process-Deliverable
Diagram
Following are the Activities and Concepts tables explaing the Process-Deliverable
Diagram from Figure 20.
78
Creative Requirements Prioritization – Activities Table
Activity
Sub-Activity
CRP Model building
Identify project-relevant
phase
stakeholders
Organize vision definition
meeting
Order Requirements
Description
Identify stakeholders relevant to the game project
at hand.
Organize project-relevant stakeholders in defining
the core creative vision for the CRM Model
Use the CRM Model in intelligently ordering
requirements.
'
Table 7: CRP’s activities table as seen in the PDD (see Figure 20).
Creative Requirements Prioritization – Concepts Table
Concept
Description
Relevant stakeholders group
The group of stakeholders responsible for building the
project’s CRP Model.
Project’s CRP Model
The CRP Model consisting of the layers comprising the
project’s core creative vision.
Ordered Product Backlog
Scrum’s Product Backlog ordered according to the
project’s CRP Model.
'
Table 8: CRP’s concepts table as seen in the PDD (see Figure 20).
First, we will take a look at building a project’s CRM Model in detail and
later we will show how this model is used in ordering (prioritizing) requirements.
Building a CRM Model
In Scrum terminology, the word prioritization is replaced with ordering, so
as to denote the need for a more intelligent, goal-appropriate prioritization.
CRP aims at delivering an ordering based on requirements relation to the
core vision of the project at hand. To do so, CRP first requires the Product Owner to identify relevant stakeholders (e.g. designers, programmers,
marketing, executives etc. – practically anyone that has to do with the core
vision) and organize a vision definition meeting. During the meeting, all
stakeholders discuss what the core vision of the game is and try to express
it in short sentences. The core vision is expressed in a game-design pitch
fashion – to the point, clear and with very few sentences (ideally between 1
Master'Thesis'Proposal'|'Aleksandar'Chervenkov'|'3318060'
and 3 sentences per layer,
depending on the complexity of the game). After
agreeing on the core vision in 1-3 sentences, they are placed as the Inner
Core (tier I) of the CRP. Further details that were not included in the Inner
Core, can be put on the secondary (less important, but still part of the core
79
vision) core definition in the Outer Core, again ideally, as short as possible.
These two layers might not be sufficient to express the core vision of a more
complex game, so the Lower Mantle (tier III) can be used, applying the
same rules. In the case of very complex games, extra layers might need to
be added to fully define the core vision of the game. Following the rules of
the CRP model (i.e. 1-3 sentences per layer), any extra elaboration on the
vision can be layered on top of the outermost layer.
It is important to remember that the CRP model represents the core vision
of the game development project and as such should focus on keeping the
model as simple and concentrated as possible, focusing on concepts and ideas
that define the core vision of the game. The empty template CRP model
in Figure 18 is shown to have 3 layers (or tiers) as a recommendation, so
that it makes the represented core vision easy to read, understand and refer
to. There is no technical reason that stops CRP model users from building
extra layers (tier 4, 5...n) , however, overcomplication of the model should
be avoided. Confining users of the model to a few layers and just a few
sentences per layer, helps them focus on what the core vision of the project
is, forcing them to leave out any details that bring little-to-no extra value to
the main idea of the project.
It is the Product Owner’s responsibility to keep the Development Team communicating with all relevant stakeholders as to deliver the right product in
the best quality. The CRP Model can be used as a means of easy communication and alignment of core vision of the product between different
stakeholders.
Since ordering the Product Backlog is exclusively the responsibility of the
Product Owner, he would be the one to apply Creative Requirements Prioritization and decide which requirements go in the next sprint cycle.
During meetings between team members (i.e. “Backlog grooming”, “Sprint
planning meeting”, “Sprint Review Meeting”, etc.) where members have
to decide what work is to be done next, the CRP Model can be used as
a means to stay focused. Moreover, if the vision has evolved in any way,
Scrum’s regular meetings are the perfect place to alter the CRP Model.
80
Ordering requirements using the CRM Model
Once the CRM Model is complete and holds the core creative vision in the
form of the three (or more) tiers, it is really straightforward to use it as a
means of ordering requirements according to it.
As we explained earlier, the core creative vision is the essential most definitive
creative vision of a game development project. Prioritizing using it means
comparing each requirement to the CRM Model (i.e. the tiered core creative
vision) and seeing if they relate to it. In other words, if any of the concepts in
the core creative vision rely on the specific requirement. One easy way to do
this is using a spreadsheet or a simple table. Using a shared spreadsheet such
as Google Sheets is a great way for the Product Owner to share the results
of the prioritization with the rest of the stakeholders and team members.
This way, they can cross-check and spot a mistake if one occurs.
In this example (see Table 9 ) we take a look at requirements prioritization
according to a 3-tiered CRM Model.
weight = 1
Tier I
weight = 1/2
Tier II
weight = 1/4
Tier III
Req. A
Req. B
Req. C
Req. D
Req. E
Req. F
Table 9: Ordering requirements using the CRM Model
1. Req. F (All three tiers rely on this requirement) 1 + ½ + ¼ = 1.75
2. Req. E - (Tiers I and III rely on this requirement) 1 + ¼ = 1.25
3. Req. B - (Tier I relies on this requirement) = 1.00
4. Req. C - (Tiers II and III rely on this requirement) ½ + ¼ = 0.75
5. Req. A - (Tier II relies on this requirement) ½ = 0.50
6. Req. D - (The core creative vision does not rely on this requirement) = 0.00
81
As one can see by looking at Table 9, each tier has its own weight assigned to
it. This weight is needed because sometimes it will be the case that different
requirements relate to more than one tiers of the core creative vision. In
the case of Req. F, all three tiers rely on it, making it the requirement with
highest importance to the core creative vision of the project. In contrast,
Req. D might have sounded like a good idea, but it should be implemented
after all the other requirements in this list, because it does not work towards
realizing the core creative vision of the project.
The ordered list of example requirements above shows how weight of each
requirement is calculated. It is easy to set up a spreadsheet to do that
automatically. What is more interesting is defining the weight of tiers. In
this example scenario, we have weighted Tier I as 1, Tier II as ½ and Tier
III as ¼, meaning Tier I is twice as important as Tier II and four times as
important as Tier III. This weighting can be done by the idea champion –
for example the Lead Designer and/or Product Owner, or some other agreed
upon stakeholder. Furthermore, it could be done during the CRM Model
building phase during the vision definition mission (see the previous section
Building a CRM Model), where it is agreed upon by all stakeholders. The
weight could be agreed upon by approximating it or stakeholders can use
one of the techniques seen in 4.3 Requirements prioritization, such as AHP,
Cumulative Voting, Numerical Assignments or Ranking depending on what
effect is desired. Since Creative Requirements Prioritization’s philosophy is
based on simplicity and ease of use, it is advisable not to spend time and
involve stakeholders in complex weight-defining procedures unless there is a
disagreement between stakeholder regarding the importance of each tier.
6.2.2
Discussion - satisfying the design criteria
As already explained earlier in this chapter, Creative Requirements Prioritization was designed according to a set of conditions (or guidelines), namely
the design criteria. Fortunately, it was possible to satisfy the full list of design criteria (see 6.1 Design criteria and rationale) when developing CRP
and the following is an elaboration on this statement.
We will start by talking about the three design criteria based on the creative
element of game development (see Design criteria in 6.1 Design criteria
and rationale; 4.5 Creative element effects on requirements prioritization).
82
Initially we set to find a metric that stimulates developer’s creative
capacity. CRP’s metric system is quite simple and measures the weight of
a requirement in relation to its importance to the core creative vision. By
definition, this helps developers focus on realizing the creative vision and
protects them from distractions and “wrong turns”.
The next in the list of criteria based on the creative element of game development – enable developer’s ability to recognize the game’s appropriateness. The CRP Model used by all stakeholders and is created early
in development during a vision definition meeting where all involved share
their thoughts and ideas and evaluate the game’s appropriateness in the specific domain it resides. This way, everyone involved in the development is
exposed to many different perspectives on the project and can better assess
the game’s appropriateness in regards to its domain.
In last criteria based on the creative element of game development, CRP
has to enable developer’s interpersonal abilities to utilize domain
knowledge, values, tools and best practices. As explained earlier,
CRP encourages group work and as such promotes, encourages and enables
developers’ interpersonal abilities to utilize domain knowledge, values, tools
and best practices.
Next, let us discuss how CRP satisfies the guidelines set by our respondent
industry experts. The first criteria – preserve the core creative vision
throughout development – is covered by the CRP Model that requires
definition, codification and preservation of the project’s core creative vision
throughout the whole development process until market delivery.
Industry experts all agreed that the CRP has to keep it flexible and introduce no substantial overhead to the development process. Both
vision definition and requirements prioritization under CRP allow for flexibility as there are very few rules that define how the process is carried out –
it is left to the involved to decide how to organize the vision definition (i.e.
could be carried online with available free collaboration tools). Moreover,
the vision definition is a one-time event in the beginning of the project, thus
it does not introduce almost any overhead to the development process. Once
the core creative vision is defined and the CRP Model built and shared (i.e.
using Google Docs), requirements prioritization can be performed at any
point in the development process by quickly comparing desired requirements
83
to the model, thus covering the next condition as well – keep it simple
and nonintrusive – use it when/if needed.
Share but don’t shift responsibility – sharing is in the roots of CRP and
even more so when it comes to responsibility. The vision definition is carried
out with all stakeholders of the project and as such, everyone from designers
and programmers to marketing and executives understands what is under
development. The requirements prioritization aspect of CRP safeguards the
vision where the goal is to realize the creative vision, thus covering the next
condition as well – take project’s creative vision into account when
deciding on features to implement.
As already discussed, the reason why the core creative vision is put in the
center of the CRP technique has to do with the shifted perspective, where
the game is not viewed as a product, but a form of art, satisfying the first
part of the next condition – view the game as an art form and not a
product – stay out the way of artistic expression. As for the second
part of the condition, CRP does not intervene artistic expression in any way
- it just provides focus.
Enable and support group creativity through shared creative vision; Provide a clear definition of the core creative vision; Communicate the core creative vision and any changes made to it during
development with all relevant stakeholders – these three conditions
are practically describing the main purpose of the Creative Requirements
Prioritization technique.
Even though the CRP technique was created with originality and art in mind,
it technically allows for the development of projects with a less original vision
like sequels to game titles or even copies. All that needs to be done is to
grasp the core creative vision of the project at hand (be it original or not) and
codify it in the CRP Model to be used during development. Thus satisfying
the condition – be useful for original as well as imitation projects.
The final condition to be satisfied was – allow all of the above, regardless
of the organizational structure of the gaming company. Neither
CRP’s vision definition nor its requirements prioritization processes rely on
a horizontal or vertical organizational structure and can be used in any
environment, as long as it allows for communication between all involved
stakeholders.
84
6.3
Evaluation
The CRP technique’s idea and design has evolved quite a lot since its first
prototype. Before the very first conversation with one industry expert (a
developer in a large mobile gaming company) we had a prototype (see appendix ) that offered a primitive and cumbersome way of prioritizing requirements, consisting of many steps and requiring a lot of attention. The initial
version of the CRP was found too heavy to use and took quite time to explain. Eventually, after it became clear this kind of feedback did not rely on
one specific respondent, the prototype was scrapped and efforts went into
finding a way to measure the ‘enjoyment’ and ‘creativity’ value in requirements and then build a prioritization method around that.
Initially, the idea was to try and see if there is a way to measure the ‘enjoyment’ value of a single requirement – an idea that sounded utopian, as
there was no literature that suggested something as subjective could be measured. However, that did not deter us from seeking an answer to the question “Is there a way to measure enjoyment in a single requirement?” with
our respondents. Responses were as expected – “this is impossible. . . and
ridiculous”, “one cannot measure the ‘enjoyment’ value of a finished product,
let alone measure the ‘enjoyment’ value of a single requirement in an unfinished game”. The surveys and semi-structured interviews had proven to be
less useful than expected, however they introduced this project’s respondent
industry experts to the research goal. The conversation with respondents
became a lot less formal, however this proved to be quite fruitful as this
marked the beginning process that would cycle around constant evaluation
of ideas and prototypes. Luckily, our design strategy (see 3 Inquiries rationale and strategy) not only allowed for this flexibility and informality but
encouraged it, as long its specific rules were followed.
Parallel to conversing with industry experts, we started looking into measuring the ‘creativity’ of a requirement. This was a vague idea but felt like
an interesting topic to research. In contrast to academic literature on measuring ‘enjoyment’, literature on ‘creativity’ was ample and soon the first
ideas of what became a prototype of the CRP as described in the previous
subsections was born. Active feedback on the prototype versions could be
expected from only 4 of our respondents. However, the final version of the
prototype Creative Requirements Prioritization technique was evaluated by
85
all but one of the 13 industry experts. With the feedback, the final changes
were made.
Despite our effort, the CRP could not be tested in a gaming company, since
smaller companies either did not have a project to test it on or as the bigger
companies had issues with sharing information about upcoming projects.
This significant part of the validation process is left as future work.
86
7
Conclusion, limitations and future work
This design research, the main part of a Business Informatics master thesis
project conducted at Utrecht University, aimed at exploring the reasons for
the absence of literature on requirements management solutions specifically
tailored for one of the most profitable (and growing) software industries –
the video game industry [7]. We set out to find what makes video games so
unique in comparison to regular business software, which enjoys a plethora
of software product management (SPM) mechanisms aiming at streamlining
and enhancing the software development process. After studying academic
literature on the matter, we discovered requirements engineering and especially requirements prioritization methods fail to address the peculiarities of
video game software development. It became clear our project could fight this
problem by locating a seriously problematic area in game development and
design an industry-specific requirements prioritization solution that would
tackle a set of the most common reasons for project failure. According to
Flood [25], all post-launch evaluations of game development point to the
same problems: development cannot keep up with the schedule, the final
product contains many defects, implemented functionalities differ from the
original ideas, producing the game took a lot of pressure, and development
required an immense number of man-hours to complete. Game development
is in many ways similar to product software development as developers follow certain software development processes. Following a poor development
method (or none at all) could result in longer development times, going over
budget and/or delivering buggy products.
What makes video games different is the creative game vision that must
be shared by the entire development team to ensure that the end product
is consistent and of good quality. Furthermore, in popular culture video
games are increasingly perceived as a form of artistic expression and even
seen as art, with its own museum exhibitions. Even research argues that “by
any major definition of art many modern video-games should be considered
art” [74][27]. It is hard to say if this shifting paradigm of perception (from
product to art) will spread industry-wide, but there are a growing number
of successful titles that were developed as art from the ground up.
87
Taking into account the aforementioned, a research question was coined:
How can a requirements prioritization technique facilitate and
nurture the creative aspects of video game software development?
In order to answer this question, our research split into two contrasting sides
– a creative side and a technical side (see Figure 1: Research scope). First,
we delved into academic literature on creativity in order to work out how it
relates to video game software. With the gained knowledge, research crossed
to the technical side and set out to find if popular requirements prioritization techniques can facilitate the creative aspect of software development.
Academic literature on requirements prioritization offered no adequate technique. Even after in-depth conversations with a dozen industry experts it
was clear that there are no requirement prioritization (RP) methods that
facilitate and/or nurture the creative aspects of video game software development. This meant that the industry effectively used no RP methods, which
could explain the most common problems encountered in game development.
At this point, our research set out to design a requirements prioritization
technique specifically tailored for the needs of game development. This is
how Creative Requirements Prioritization (CRP) - the main deliverable of
this thesis project - was born. Using gathered knowledge from relevant academic literature research and combining it with empirical research in the
form of conversation and collaboration with industry experts, we were able
to construct a list of design criteria (see 6.1 Design criteria and rationale)
that would serve as the blueprint for the development of the CRP – a twocomponent technique consisting of the CRP Model responsible for the definition and preservation of a shared core creative vision and the CRP method
for intelligent ordering of requirements based on evaluating requirement’s
importance to the core creative vision.
The main limitation of our research has to do with our unsuccessful attempt
to carry our research inside a gaming company and test the Creative Requirements Prioritization technique with a real game development project.
Even though we were able to collaborate with industry experts (designers,
programmers, writers and artists) on evaluating and validating the proposed
solution, a proper validation would require real world testing – something
that is left as future work of this research. Respondents’ positive feedback
on CRP is uplifting, however one should use this technique with caution,
88
keeping in mind that is the product of an explorative design research in a
field where no such other solution exists. The author encourages anyone willing to carry this research further and test and evolve Creative Requirements
Prioritization to do so and both enlighten this dark corner of academic literature as well as enhance the development process in one of the most exciting
and profitable software industries.
89
References
[1] ASQ
-
Service
Quality
Division;
retrieved
from:
http://asq.org/service/body-of-knowledge/tools-analytic-hierarchyprocess.
[2] Beck, Kent; et al. (2001). Manifesto for Agile Software Development.
Agile Alliance. Retrieved 10 April 2012.".
[3] Gamasutra - News - Minecraft Draws Over $33 Million In Revenue From
1.8M Paying Customers.
[4] The Video Game Revolution: The History of Games | PBS.
[5] Study: Average development costs as high as $28m Study: Average
development costs as high as $28m Study: Average development costs
as high as $28m Study: Average development costs as high as $28m,
2010.
[6] David R. Anderson, Dennis J. Sweeney, Thomas A. Williams, and Mik
Wisniewski. An Introduction to Management Science: Quantitative Approaches to Decision Making. Cengage Learning, 2008.
[7] ESA The Entertainment Software Association. Industry Facts, 2011.
[8] James Bach. The Challenge of "Good Enough" Software. Quality, pages
1–11, 1996.
[9] P. A. Bachelor and W. B. Michael. The structure of intellect model
revisited. In Robert J Sternberg, editor, Creativity research handbook,
pages 155–182. Cambridge University Press, 1997.
[10] Bob Bates. Game Design. Premier Press, 2004.
[11] Kent Beck. Extreme Programming Explained: Embrace Change. The
XP Series. Addison-Wesley, 1999.
[12] Patrik Berander. Evolving Prioritization for Software Product Management. Engineering, page 250, 2007.
[13] Erik Bethke. Game Development and Production. Wordware Publishing,
Inc., 2003.
90
[14] M Boden. Computational models of creativity. In Handbook of creativity, volume 351, pages 351–373. Cambridge University Press, 1999.
[15] S Bowler, T Donovan, and DM Farrell. Party strategy and voter organization under cumulative voting in Victorian England. Political Studies,
47(5):906–917, 1999.
[16] S Bradner. RFC 2119. Internet24 November 2004, 1997.
[17] Alan Bryman and Emma Bell. Business Research Methods, volume 2nd
of Introducing qualitative methods. Oxford University Press, 2007.
[18] Heather Maxwell Chandler. Game Production Handbook. Infinity Science Press, 2009.
[19] M Csikszentmihalyi. Society, culture, and person: A systems view of
creativity. In R J Sternberg, editor, The Nature of Creativity, chapter 13, pages 325–339. Cambridge University Press, 1988.
[20] Mihaly Csikszentmihalyi. Creativity: Flow and the Psychology of Discovery and Invention. Harper Perennial, 1997.
[21] Mihaly Csikszentmihalyi. Implications of a systems perspective for the
study of creativity. In Robert J Sternberg, editor, Handbook of Creativity, number 1963, chapter 16, pages 313–335. Cambridge University
Press, 1999.
[22] A M Davis. The art of requirements triage. Computer, 36(3):42–49,
2003.
[23] N. Fielding and R. Lee. Using Computers in Qualitative Research. Sage
Publications, 1991.
[24] R. A. Finke. Mental imagery and visual creativity. In M A Runco, editor, Creativity research handbook, volume 351, pages 183–202. Cresskill,
NJ: Hampton., 1997.
[25] K Flood. Game Unified Process, 2003.
[26] John P Flynt and Omar Salem. Software Engineering for Game Developers. Technology, page 862, 2005.
91
[27] James Paul Gee. Why Game Studies Now? Video Games: A New Art
Form. Games and Culture, 1(1):58–61, 2006.
[28] Alan Gershenfeld, Mark Loparco, and Cecilia Barajas. Game Plan:
The Insider’s Guide to Breaking In and Succeeding in the Computer
and Video Game Business. {St. Martin’s Griffin}, 2003.
[29] H.E. Gruber and D.B. Wallace. The case study method and evolving
systems approach for understanding unique creative people at work. In
R. J. Sternberg, editor, Handbook of creativity, pages 93–115. Cambridge
University Press, 1999.
[30] J. P. Guilford. Creativity. APA Presidential Addresses, 5:444–454, 1950.
[31] Wolfgang Hamann. Goodbye Postmortems, Hello Critical Stage Analysis, 2003.
[32] Alan R Hevner, Salvatore T March, Jinsoo Park, and Sudha Ram. Design Science in Information Systems Research. MIS Quarterly, 28(1):75–
105, 2004.
[33] IEEE-Computer-Society. IEEE Recommended Practice for Software Requirements Specifications. IEEE Std 830-1998, 1998.
[34] J Karlsson, C Wohlin, and B Regnell. An evaluation of methods for
priorizing software requirements. Information and Software Technology,
39:939–947, 1998.
[35] L Karlsson, T Thelin, B Regnell, P Berander, and C Wohlin. Pairwise comparisons versus planning game partitioningexperiments on requirements prioritisation techniques. Empirical Software Engineering,
12(1):3{\textendash}33, 2007.
[36] Albert N Katz. Creativity and the cerebral hemispheres. American
Psychologist, 34:279, 1979.
[37] James C Kaufman and Robert J Sternberg. The Cambridge Handbook of
Creativity. Cambridge Handbooks in Psychology. Cambridge University
Press, 2010.
92
[38] Kyung Hee Kim. Can We Trust Creativity Tests? A Review of the Torrance Tests of Creative Thinking (TTCT). Creativity Research Journal,
18(1):3–14, 2006.
[39] Soren Lauesen. Software Requirements: Styles & Techniques. AddisonWesley Professional, 2002.
[40] Dean Leffingwell and Don Widrig. Managing Software Requirements.
Addison Wesley, 2003.
[41] Laura Lehtola and Marjo Kauppinen. Empirical Evaluation of Two
Requirements Prioritization Methods in Product Development Projects.
Software Process Improvement, 3281:161–170, 2004.
[42] Laura Lehtola and Marjo Kauppinen. Suitability of requirements prioritization methods for market-driven software product development.
Software Process: Improvement and Practice, 11(1):7–19, 2006.
[43] Laura Lehtola, Marjo Kauppinen, and Sari Kujala. Requirements Prioritization Challenges in Practice. Product focused software process improvement, 3009:497–508, 2004.
[44] Yvonna S Lincoln and Egon G Guba. Naturalistic inquiry, volume 23
of Sage focus editions. Sage Publications, 1985.
[45] Neil A M Maiden and Cornelius Ncube. Acquiring COTS software
selection requirements. IEEE Software, 15(2):46–56, 1998.
[46] John Markoff. Atari acts in an attempt to scuttle software pirates.
InfoWorld, pages 28–29, 1981.
[47] Colin Martindale. Biological bases of creativity. In Robert J Sternberg,
editor, Handbook of Creativity, chapter 7, pages 137–152. Cambridge
University Press, 1999.
[48] Morgan McGuire and Odest Chadwicke Jenkins. Creating Games: Mechanics, Content, and Technology. A K Peters, 2009.
[49] Michael E. Moore and Jeannie Novak. Game Development Essentials:
Game Industry Career Guide. Cengage Learning, 2010.
93
[50] Michael D Mumford. Where Have We Been , Where Are We Going ?
Taking Stock in Creativity Where Have We Been , Where Are We Going
? Taking Stock in Creativity Research. Creativity Research Journal,
15(773566367):107 – 120, 2010.
[51] An Ngo-The and Günther Ruhe. Decision Support in Requirements
Engineering. Engineering and Managing Software Requirements, pages
267–286, 2005.
[52] John M. Nicholas and Herman Steyn. Project Management for Engineering, Business, and Technology. A Butterworth-Heinemann Title,
2011.
[53] Object Management Group OMG. UML 2.2 Superstructure Specification (formal/2009-02-02), 2009.
[54] Mr Kevin Oxland. Gameplay and Design. Addison Wesley, 2004.
[55] Michael Quinn Patton. Qualitative research and evaluation methods.
SAGE, 2002.
[56] Bernard Perron and Mark J.P. Wolf. The Video Game Theory Reader
2. Routledge, 2009.
[57] Duncan J. Petrie. Creativity and Constraint in the British Film Industry. Palgrave Macmillan, 1991.
[58] J A Plucker and J S Renzulli. Psychometric approaches to the study of
human creativity. In Robert J Sternberg, editor, Handbook of creativity,
chapter 3, pages 35–61. Cambridge University Press, 1999.
[59] E. Policastro and H. Gardner. From case studies to robust generalizations: An approach to the study of creativity. In R. J. Sternberg,
editor, Handbook of creativity, pages 213–225. Cambridge University
Press, 1999.
[60] Rob Pope. Creativity: Theory, History, Practice. Routledge, 2005.
[61] C. P. Puri. Agile Management: Feature Driven Development. Global
India Publications Pvt Ltd, 2009.
94
[62] Björn Regnell, Martin Höst, Johan Natt Och Dag, Per Beremark, and
Thomas Hjelm. An Industrial Case Study on Distributed Prioritisation
in Market-Driven Requirements Engineering for Packaged Software. Requirements Engineering, 6(1):51–62, 2001.
[63] E M Rogers and Judith K Larsen. Silicon Valley Fever: Growth of High
Technolgy Culture, 1984.
[64] M A Runco. The creativity research handbook. Hampton Press, 1997.
[65] Thomas L. Saaty and Luis G. Vargas. Models, Methods, Concepts &
Applications of the Analytic Hierarchy Process (International Series in
Operations Research & Management Science). Springer, 2000.
[66] Thomas Lorie Saaty. Analytic Hierarchy Process. McGraw Hill Higher
Education, 1980.
[67] Motoshi Saeki. Embedding metrics into information systems development methods: An application of method engineering technique. In Advanced Information Systems Engineering, page 1031. Springer, Springer,
2003.
[68] R. Keith Sawyer. Explaining Creativity: The Science of Human Innovation. Oxford University Press, USA, 2006.
[69] Ken Schwaber. Agile Project Management with Scrum (Microsoft Professional). MICROSOFT PRESS, 2004.
[70] Thomas A Schwandt. Three Epistemological Stances for Qualitative
Inquiry: Interpretivism, Hermeneutics, and Social Constructionism. In
Norman K Denzin and Yvonna S Lincoln, editors, Handbook of Qualitative Research, volume 2 of The Landscape of Qualitative Research:
Theories and Issues, chapter 7, pages 189–213. Sage, 2000.
[71] John V Seidel. QDA : A Model of the Process Noticing , Collecting ,
Thinking about Things. Analysis, (c):1–15, 1998.
[72] D. K. Simonton. Historiometric studies of creative genius. In M. A.
Runco, editor, Creativity research handbook, pages 3–28. Cresskill, NJ:
Hampton, 1997.
95
[73] D. K. Simonton. Creativity from a historiometric perspective. In R. J.
Sternberg, editor, Handbook of creativity, pages 116–136. Cambridge
University Press, 1999.
[74] Aaron Smuts. Are Video Games Art ? Contemporary Aesthetics, 3:1–16,
2005.
[75] Ian Sommerville and Pete Sawyer. Requirements Engineering: A Good
Practice Guide. John Wiley & Sons, 1997.
[76] Robert J Sternberg. Handbook of creativity. Perspectives on individual
differences. Cambridge University Press, 1999.
[77] V Vaishnavi and W Kuechler. Design Research in Information Systems.
Information Systems Journal, 22(520):209–233, 2004.
[78] Inge Van De Weerd, Sjaak Brinkkemper, Richard Nieuwenhuis, Johan
Versendaal, and Lex Bijlsma. On the Creation of a Reference Framework
for Software Product Management: Validation and Tool Support. 2006
International Workshop on Software Product Management IWSPM06
RE06 Workshop, 0:3–12, 2006.
[79] Inge Van De Weerd, Stefan De Weerd, and Sjaak Brinkkemper. Developing a Reference Method for Game Production by Method Comparison. Situational Method Engineering Fundamentals and Experiences,
244(244):313–327, 2007.
[80] T B Ward, S M Smith, and R A Finke. Creative Cognition, volume 79 of
Theory, Research, and Applications. Cambridge University Press, 1999.
[81] Peter Watson. Ideas: A History of Thought and Invention, from Fire
to Freud. Harper, 2005.
[82] Inge Van De Weerd, Sjaak Brinkkemper, Jurriaan Souer, and Johan
Versendaal. A situational implementation method for web-based content
management system-applications: method engineering and validation in
practice. Software Process: Improvement and Practice, 11(5):521–538,
2006.
[83] R W Weisberg. Creativity and knowledge: A challenge to theories. In
R J Sternberg, editor, Handbook of creativity, volume 227, chapter 12,
pages 226–250. Cambridge University Press, 1999.
96
[84] Robert W. Weisberg. Creativity: Understanding Innovation in Problem
Solving, Science, Invention, and the Arts. Wiley, 2006.
[85] Jason Whittaker. The cyberspace handbook. Media practice. Routledge,
2004.
[86] K. Wiegers. Software Requirements (Dv-Best Practices). Microsoft
Press,U.S., 1999.
[87] Liang Zeng, Robert Proctor, and Gavriel Salvendy. Can Traditional
Divergent Thinking Tests Be Trusted in Measuring and Predicting RealWorld Creativity? Creativity Research Journal, 23(1):24–37, 2011.
[88] Vera L. Zolberg. Constructing a Sociology of the Arts. Cambridge University Press, Cambridge, 1990.
97
Appendix
Initial prototype version of the CRP method
Early version of the CRP. Newer version is under development, incorporating
initial feedback from respondents and findings in literature.
The Creative Requirements Prioritization method
The goal of this game-oriented method for prioritizing requirements is to
prevent scheduling problems, scope problems, crunch time and technological
problems and take into account the creative aspects of game production.
Stage I
This method assumes that there already is a list of all potential requirements
for the game, for example, written down in a conceptual document.
Stage II
The second stage of the prioritization process is based on the triage method.
Requirements are roughly divided into three categories, namely the musthaves, definitely exclude and nice-to-haves. The must-haves are critical requirements that must be implemented. The definitely exclude are eliminated.
This will leave a pool of nice-to-haves requirements which will be assigned
priorities in the third stage. Based on the ranking system presented below,
some of them will be implemented while others will be dropped out.
Stage III
This is the core of this “Creative Requirements Prioritization” method, which
consists of 3 steps:
? Estimate the Cost of nice-to-have requirements
? Estimate their corresponding Entertainment Value
? Calculate CRP index for all nice-to-have requirements
A
step 1
In the first step, an estimated cost of nice-to-have requirements is determined
for each of the requirements. This should be a rough estimation of the
currency value of man-hours required to fully implement a requirement (this
includes programming, acting, recording, etc.). All those costs are then
summed up in order to create a total cost of all the nice-to-have requirements.
step 2
This is where the CRP method uses a customized variant of the 100-point
method.
In this stage, each stakeholder (e.g. developers, designers, product manager
etc.) divides 100 points over the various requirements that they think will
deliver the most fun to players. Once those points have been distributed, the
values will be scaled for every requirement. This scaled point value will be
called the entertainment value (EV). We calculate the EV with the following
formula:
EV of requirement x =
(total cost of all requirements / total points distributed over all
requirements) * points distributed to requirement x
Each requirement has now been assigned an entertainment value as an amount
of the total cost.
step 3
Finally, in the third step, the CRP method calculates the CRP index for
all nice-to-have requirements. The CRP index is the cost of the requirement
minus the requirement’s EV.
Stage IV
At this point, the requirements have been given a priority, they can be
assigned the appropriate amount of resources required for implementing each
requirements, and can finally be included in the Game Design Document.
B