User Requirements Analysis for Educational
Games in Manufacturing
Jannicke Baalsrud Hauge1, Heiko Duin1, Manuel Oliveira2, Klaus-Dieter Thoben1
1
BIBA, Hochschuring 20, D-28359 Bremen, Germany, {baa, du, tho}@biba.uni-bremen.de;
2
Alfamicro, Alameda da Guia, N.192A, 2750 Cascais, Portugal, [email protected]
Abstract
Computer games become more and more important, as a tool to support education and training at school and
university, as well as vocational training in industry. Contrary to the development of business software, the design of
a computer game has to follow other development principles, specifically when collecting the end user (potential
gamer) requirements. This paper presents two different approaches used to collect end user requirements for the
development of computer games to be applied as vocational training tool in the manufacturing industry. These
approaches are compared with each other, and analyzed against traditional requirements methods from software
engineering. The paper presents the result of the analysis, along with the lessons learnt.
Keywords
User requirements analysis, Computer games, Educational gaming, Vocational training, Serious Games
1
Introduction
One of the consequences of being part of a dynamic organisation is that the employee faces an
ever-changing working environment. This raises a need of continuous learning for both
employees [OECD 2004] and organisations [Schwesig 2004]. During the last decade e-Learning
approaches were investigated and deemed suitable for mediating skills in vocational training
[Echelmeyer 2005]. Nevertheless, it has been stated (e.g. in [Windhoff 2001], [Schwesig 2005],
[Thoben 2005]) that there are some problems when considering the mediation of soft skills (like
communication and collaboration skills) on a theoretical level. Serious computer games (ie:
business games) would in many cases, enable the mediation of these skills [Thoben 2005].
The usefulness of an educational game for a certain company depends on whether the user
requirements on which the game is based, match those of the company or not. Therefore, it is
essential to involve the potential target group from an early stage of the project. Additionally the
success of a game depends on its adaptability and portability, hence the need to develop a generic
adaptable model and tool (see [Thoben 2005] and [Baalsrud Hauge 2004]).
Being involved in several projects aiming at designing and developing educational games (like
COSIGA [Pawar 1995], SHARE [Schwesig 2005], SPIKO [SPIKO 2006] and PRIME [PRIME
2006]), the authors have tried to figure out which methods and approaches for user requirement
engineering deliver the best result taking into account limited monetary resources and a short
time. This article will focus on the experience gained from the user requirements analysis for the
two most recently running projects.
1.1
Project SPIKO
The SPIKO1 project, which is a German founded project, aims at developing a computer based
game for mediating collaboration skills. The target user group are employees with higher
1
SPIKO is the abreviation for the project long title: SPIelend unternehmensübergreifende KOoperationen erleben
und erlernen. It is a German project funded by BMBF ( Ministry of Education and Research)
education performing collaboration. The objective of the game is to qualify the employees for
working in a collaborative environment by increasing their competencies. An important aspect of
the project was to support sufficient flexibility for the game to be ported to different industries.
The project started 11/2003 and will end in 10/2006. The user requirements analysis within this
project has been completed by 01/2005.
1.2
Project PRIME
The main objective of the PRIME2 project is to give business professionals a learning
environment where they can experiment with new ideas and learn how to handle the entire life
cycle of products and processes for different stakeholders within an organization. The PRIME
project proposes to achieve this by enhancing current working environments with a new
paradigm based on serious gaming. This will provide the means for learning by experience
within a virtual environment that is safe and foments risk taking without detrimental impact on
the business – the Virtual Business Environment (VBE). The experience gained by the user is
based on strategic management, including multi-stakeholder negotiation and business
connectivity. However, unlike SPIKO, the aim was not to create an educational curriculum.
As the user requirements analysis within the SPIKO project was already completed, when
PRIME started in 09/2005, the lessons learnt in SPIKO were used as a starting point for the
design of the user requirements analysis task.
2
2.1
Relation to Existing Theories and Work
User Requirements Analysis in Software Engineering
There are numerous different development methodologies (see [Wiering 1998]) to choose from,
but none is able to guarantee the success of a project. This supports the claim “silver bullets” do
not exist (see [Brooks 1987]) to solve the nightmares associated to the software process of
complex systems. It is widely accepted that software engineering implies more than sophisticated
techniques (e.g. [McConnell 1998]), which is the only way to tackle the complexity of large
scale systems that are inherently multi-disciplinary [Grimson 2000].
Although different, all methodologies begin with the elicitation of user requirements, their
analysis and subsequent specification. The Requirements Engineering (after [Zave 1997]) is a
dedicated branch of software engineering aimed at developing precise and effective specification
of user requirements. The problems of understanding during the elicitation process can lead to
requirements that are ambiguous, erroneous, incomplete and inconsistent, which do not address
the end-user needs. To address the problems, different elicitation techniques exist to assist the
process and counter the risks, which according to [Nuseibeh, Easterbrook 2000] can be
categorized into the following classes: Traditional, Group-based, prototyping (see [Davis 1992]),
model-driven ([Lamsweerde et al 1998] and [Maiden 1998]), cognitive and contextual.
It is common practice by the developer community that user requirements are categorized as
functional and non-functional. The former encompasses all the visible aspects of the system, in
particular the inputs and respective outputs, whilst the latter comprises non-visible constraints,
such as performance, dependability, extensibility, reusability and maintainability. An alternative
wording is behavioural requirements (functional) and development quality attributes (nonfunctional).
The results from industry studies carried out in the 70s [Boehm 1981], which has been replicated
several times since then, clearly indicate that the later in the development process an error is
detected, the higher the cost incurred to correct it. In fact, with traditional software development,
2 PRIME is the short name for the project Providing Real Integration in Multi-disciplinary Environments and is
funded under IST/NMP in FP6.
40-60% of the system errors have been traced back to the requirements and analysis phase, with
an additional 70-85% of the revisions being attributed to requirements errors [Leffingwell 1997].
As a result, the requirements are extremely important to the success of the software product.
The supporting mechanism to the development process of a game is the design document, which
conveys a written blueprint of the final instantiation of the game. The game design document
compounds together the user requirement documentation, functional and non-functional
specifications characterised in the classic waterfall software development methodology [Royce
1987].
2.2
Requirements Analysis for Computer Games
The development of games does not entail any formal user requirements, as the approach is
driven by the vision of the developer team concerning an entertainment product that has market
potential. In fact, although the development is driven by what a user will potentially want in the
end product, the user is normally involved much later in the development process when testing
and evaluation is required.
The aim of both projects, SPIKO and PRIME, is to develop a game, which not only is engaging,
but also addresses actual user needs. As such, unlike the usual approach to developing games, it
is not enough to have a well-defined vision owned by the developers. It is necessary to identify
and analyse any requirements that the users have. However, this presents an interesting challenge
to the development process, as the particular users in question have little or non-existent
experience with games, thus unable to envision how the game could address their needs.
3
Research Approach
3.1
The SPIKO Approach to User Requirements Analysis
Even though the partners had been knowing each other for years and being cooperating in
previous projects, it was clear that the developers’ knowledge of running processes at the end
users was not good enough for being able to design a game with realistic scenarios. The purpose
of the SPIKO end user requirements approach was therefore to provide a guide to how the
SPIKO development team could gather and capture the individual requirements of the end users.
Since the main outcome of the project is an educational game, it is not enough to only focus on
generating realistic scenarios, but also find out the educational needs of end-user. The
consortium agreed on an iterative approach with a mix of interviews partly using questionnaires
and business process modelling. The methodology to be used is summarised as follows:
Phase 1:
•
Initial visits to each of the end users
•
End users decide upon relevant collaboration scenarios
• End users defines target group and didactical objective
Phase 2:
•
Establish common template to gather data flow information
•
Conduct end user requirement sessions and document requirements gathered in common
template using appropriate modelling diagrams
Phase 3
•
Consolidate user requirements – identifying generic components
•
Defining a generic process model
The approach of SPIKO followed a waterfall-based paradigm in the development process. In
such an approach it is often difficult to go back to the beginning, when at a later phase, there is a
request to introduce changes. This requires end users to have a clear idea of what they want so
they may state their needs so the developer team may successfully capture those requirements.
However, this is usually not the case.
3.2
User Requirements Analysis Results of the SPIKO Project
Business process analyses were carried out at the end users in order to find the relevant
cooperation processes, as well as, to use them for identifying the end user requirements. These
processes are the basis of the SPIKO game. All three end users had a high degree of similarity in
the initiating process of collaboration, so a generic model was applicable. For the second relevant
phase, the operation of the collaboration, the partner processes were more individual, partly
related to the different type of collaboration they had and partly reflecting the industrial sector.
The requirement analysis showed that it was necessary to design a dynamic gaming
environment, which would allow the companies to change the scenarios according to the
changing business and working environment.
The main end user requirements on the game were:
•
Should have an editing tool for generation of new scenarios.
•
Offers different levels and main foci (choice at the beginning of a game).
•
The user can choose between different scenarios. For each phase of a scenario, training
material is available.
Derived from the interviews, the following end-users’ learning requirements on the game were
identified: exemplified learning by experience, training of soft and process skills, acquisition of
background knowledge, playing in a realistic environment, anticipation of the mode of action,
the potential and the boundaries of enterprise collaboration and playing different collaboration
scenarios. Based upon these requirements, the expected outcome is a tool with three different
scenarios, each with several different cases with adjustable parameters. Each case is based upon
a structured model.
3.3
Pros and Cons of the SPIKO Approach
Analysis of the SPIKO approach has shown that the very intensive “as-is” process analysis gave
the developers an excellent knowledge of relevant processes for each end user. This was
important for designing and developing the scenarios of the game, but it was far too much time
consuming. Furthermore, using so much time on the process analysis left less time for discussing
and defining the didactical objectives, and sub-objectives, with each end user. Consequently, the
resulting curriculum has some quite generic didactical objectives. This became evident when
software development process began in earnest. Additionally, it was revealed that the process
analysis had generated unrealistic end-user expectations concerning the game, which could not
be implemented with the allocated budget and time frame. This was compounded by the lack of
experience with educational games by some of the end users. In hindsight, most of the
misunderstandings could have been diverted with user expectation management. One way to
achieve this would have been:
•
To dedicate more time in developing the didactical objectives with the end users, instead
of expecting them to know what they want in advance;
• Delimit the boundaries of the simulation game.
However, the collection of the end users requirements over such a long time had clearly showed
that it is necessary to design a game that is adaptable to the ever changing of the end users’
requirements.
SPIKO User Requirements Analysis
•
•
•
•
Advantages
Allow external partners a good understanding of
company activities and running processes
High involvement and engagement of end users
Ensures a user centric design
Is flexible and adaptable. Allows for changes in
the requirements
•
•
•
•
Disadvantages
The end users created a high expectation on
what a game could mediate, how a game could
be used and mirroring end user specific
processes very detailed.
Time consuming approach
Too generic curriculum after requirements
analysis
Difficult to find a generic gaming environment
(too user specific)
Table 1: Advantages and disadvantages in the user requirements analysis approach in SPIKO
3.4
The PRIME Approach and User Requirements Analysis
The PRIME project takes a serious game approach where the final user experience is not based
on a web based hyperlinked product, but a rich graphical user interface. However, this raised
interesting challenges on how to process the user requirement analysis, considering the little
awareness in the end-users of what a serious game is, let alone of issues such as game play and
game design.
Figure 1: The PRIME development approach
As a result, in PRIME, a user-centred development approach was taken, using principles from
the Agile Programming Community after [Beck et al 2006] and the spiral development approach
(see [Boehm 1988]). The ensuing methodology is illustrated in Figure 1, where the major
milestones are indicated every sixth month for the total duration of the project. The user
requirements analysis is not a well-defined stage in the methodology, so the end-users provide
input into the design and implementation process from the very beginning until the beta stage
milestone. This is different to other game design methodologies, which follow a waterfall
approach with phases, e.g. definition phase, design phase, realisation phase and implementation
phase (after [Hijden 2005]).
The initial step was to promote awareness of serious games with all types of end-users: industry,
university, research institute and ICT providers. To achieve this, a workshop on serious games
was organized at the project kick-off meeting. The workshop not only defined serious games, but
discussed impact on organizations/employees and an overview of existing serious games. From
the onset of the project, it was decided that the aim was not to create a customised PRIME to
each end-user, but a generic one that all would use. This has encouraged the end-users to actively
engage the design process to ensure their interests during the process of generalization across the
different organizational strategies and structures.
Underlying the entire development methodology and processes is the project’s Vision, which
evolves as the project unfolds. The Vision is fundamental to convey and keep a consolidated
drive of the project, considering the different interests and roles of the participants. Embedded
into this methodology, the end user requirements analysis addressed the following objectives:
•
Analysis of the current vocational practice and how serious gaming could fit into.
•
Understanding of end users business objectives, strategies, products and processes
including a stakeholder analysis.
•
Understanding of the strategic issues connected with the business of each of the end user.
•
Analysis of who will be the gamer and what are their learning objectives.
• Identification of technological and other requirements.
The fieldwork of user requirements analysis was performed by user-developer-partnerships: each
end user was supported by at least one of the developers in the project. A guideline has been
developed to ensure that each of the partnerships focus on the same topics in the same manner.
4
4.1
Findings
User Requirements Analysis Results of the PRIME Project
The initial results of the end user requirements analysis as presented in the following paragraphs
are not the final requirements. Due to the spiral approach taken in PRIME, the requirements will
evolve as the project is moving on.
The analysis revealed that only one of the six end users has collected experience with the usage
of games in vocational training. Therefore, like in the SPIKO project, the end users expected to
have a realistic simulation of their own business. However, this was quickly detected and a
strong process of controlling end-user expectations was put in place from the start, so end-users
are aware that the focus is to create a believable experience where the learning is transferable to
reality.
The business cases, where the single end users are interested to be implemented by the PRIME
serious game are dealing with the market introduction of new and complex products, and
different flavours of product management. All of the end users provided rough flowcharts, how
the business processes are organised within their organisation. The strategic issues connected
with their business cases are make or buy decisions, customer order decoupling point decisions,
delivery network level considerations, chasing versus stocking production plans, and the
integration of stakeholders and suppliers.
The gamers within each of the end user organisations will always be medium level management
staff. The learning objectives for these employees are currently focussing on the mediation of
business processes skills. This is very much in line with the end user wishes to have their own
organisation exactly modelled by the game, but such can be achieved by customization by each
end-user.
The technological constraints for implementing a game at the end user organisation are low. A
game client embedded within a web browser is preferred, but another technological solution can
be accommodated provided the security policy is followed.
4.2
Advantages and Disadvantages of the PRIME Approach
The spiral user-centred development methodology adopted in PRIME has ensured that the
industrial end-users are highly motivated and engaged in the project, participating with avid
enthusiasm. This has facilitated the process of eliciting the user requirements and the activities
concerning the design of the PRIME serious game.
In PRIME, the design has not waited until the elicitation of the user requirements and their
subsequent analysis. Quite on the contrary, the design process has initiated and runs in parallel
with the user requirements, using tools such as scenarios and storyboarding to facilitate the
brainstorm and exchange of knowledge. In these initial months of the project, the user
requirements have assumed a predominant role, albeit strongly influenced by the design
processes such as the Vision Definition, Concept and Game Design (all three supported by
documentation). The predominance will change, with the user requirements becoming less
predominant, but still provide valuable input to the design and implementation processes. The
advantages and disadvantages of the PRIME approach to the user requirements analysis is
summarised in the following table.
PRIME User Requirements Analysis
•
•
•
•
Advantages
Disadvantages
High involvement and commitment of the end- • Requires good communication with all
participants in the development methodology
user
A truly user centred design where the user is • Requires good support to exchange knowledge
involved from the onset of the project
Reduced risk of the final product not matching
the user expectations
Promote awareness of the product within the
end-users before final release
Table 2: Pros and cons in the user requirements analysis approach in PRIME
4.3
Enhancements in comparison to SPIKO
In comparison to the advantages and disadvantages of the approach taken in SPIKO, the
approach of PRIME was able to overcome most of the pitfalls. The enhancements observed so
far can be condensed into the following two observations:
•
A traditional user requirements analysis will most likely result in end users wanting to have
an exact simulation of their own organisation and business environment. Due to the spiral
approach taken in PRIME, coupled with tough control mechanisms to manage end-user
expectations, the learning objectives move more towards mediating the intangible soft skills
connected with their business.
•
It is good practice not to spend too much time on detailed process analysis at end users, but
to have a quick start to understand the end users objectives, businesses, processes etc. After
that, the requirements are continuously refined between developers and end-users by
commonly validating gaming ideas against end user expectations. This reduces risk to miss
end user expectations because these expectations are continuously developed towards what
games can mediate.
5
Conclusions
Based upon the experience garnered with user requirement analysis for educational games, the
use of traditional software development approaches is not appropriate. This is due to the fact that
end users’ seldom have a clear understanding of educational game and a precise idea of their
didactical requirements. The use of educational games is particularly suitable for mediating
intangible knowledge through the experience. However, this can only be achieved with abstract,
but user specific learning objectives.
It is very important to guide end users through user requirement analysis and to inform them at
an early stage of the advantages and limitation of the simulation underlying the game. Also, the
didactical objectives and sub-objectives should be intensively discussed and stated early in the
project phase. These are to be correctly and clearly defined from the onset of a project.
In conclusion, the spiral approach based on agile and extreme programming principles for the
user requirement analysis, as carried out in the PRIME project, is a more suitable approach for
development of educational games than the traditional approach. However, there is a need to
carry out further research to provide evidence supporting a generalization of the PRIME success
to other similar projects based on the development of educational games.
Acknowledgements
The authors thank the respective project partners, the DLR and the German Ministry of education and research
(BMBF) as well as the European Commission for all scientific, organisational and financial support. The SPIKO
project is funded by the programme: designing work in virtual enterprises. The PRIME project is funded under
contract number FP6-016542 within the Sixth Framework Programme, Priority IST/NMP.
References
Baalsrud Hauge, J.et al: Business games- an effective tool for experiencing collaboration in production networks. pp.30-37. In:
Smeeds, R. et al: Experimental interactive learning in industrial management.: new approaches to learning, studying and
teaching. Proc. of the 9.th Wrkshp. of IFIP W.G 5.7 SIG on exp. Interactive Learning in Industrial Mngmnt., Espoo, 2005.
Beck, K. et al: Manifesto for Agile Software Development. http://www.agilemanifesto.org. WWW page accessed 12.1.2006.
Boehm, B.: Software Engineering Economics. Prentice Hall, New Jersey. 1981.
Boehm, B.: A Spiral Model of Software Development and Enhancement. Computer, Vol. 21, No. 5, May 1988.
Brooks, F.: No Silver Bullet: Essence and Accidents of Software Engineering. Computer, Vol.20, N.4, April 1987.
Davis, A.: Operational Prototyping: A New Development Approach. Software, Vol. 9, N.5, 1992.
Echelmeyer, W.; Nagels, D.; Gavirey, S.: Neue Wege in der prozessintegrierten Anpassungsqualifizierung. Band 4. Verlag Dr.Ing. Paul Christiani GmbH&Co.Kg. Konstanz, 2005. p. 53.
Goodrich, V., Olfman, L.: An Experimental Evaluation of Task and Methodology Variables for Requirements Definition Phase
Success. Proceedings of the Twenty-Third Annual Hawaii International Conference on System Sciences, January 1990.
Grimson, J., Kugler, H.: Software Needs Engineering – a Position Paper. Proceeding of ICSE. Limrick, 2000.
Hijden, Pieter van der: On the Development of Workflow-based Asynchronous Group Simulations. Proceedings of the 36th
Annual Conference of the ISAGA. 27 June - 1 July 2005. Atlanta, GA. (WWW page http://isaga05.gatech.edu/pap/p104/
ISAGA%202005%20paper.doc)
Lamsweerde, A., Darimont, R., Letier, E.: Managing conflicts in goal-driven requirements engineering. IEEE Transactions on
Software Engineering, Vol. 24, N. 11, 1998.
Leffingwell, D.: Calculating the Return on Investment from More Effective Requirements Management. American Programmer,
Vol.10, N.4.
Maiden, N.: CREWS-SAVRE: Scenarios for Acquiring and Validating Requirements. Automated Software Engineering, Vol. 5,
N. 4, 1998.
McConnell, S.: The Art, Science, and Engineering of Software Development. IEEE Software, Jan/Feb 1998.
Nuseibeh, B., Easterbrook, S.: Requirements engineering; A Roadmap. Conference on The Future of Software Engineering.
Limerick, 2000.
OECD Observer, February 2004 , Policy brief, p.2
Pawar, K. S., Thoben, K.-D., Oehlmann, R.: Developing concurrent engineering conceptual model and knowledge platform. In:
Proceedings of the Second Conference on Concurrent Engineering, Research and Application (CERA). USA: Washington
DC. 23–25 August, 1995. pp. 487–497
PRIME: PRIME, http://www.alfamicro.pt/prime/. WWW page accessed 25.3.2006.
Royce, W.: Managing the Development of Large Software Systems. Proceedings, 9th International Conference on Software
Engineering, Monterey, March, 1987.
Schwesig, M.: Development of a web based management simulation of knowledge exchange in networked manufacturing
organisation. Bremer Schriften zu Betriebstechnik und Arbeitswissenschaft, Nr.54, Verlag Mainz. 2005.
SPIKO: SPIKO. http://www.spiko.org. WWW page accessed 7.1.2006.
Thoben et al.: Training through gaming: applying a simulation based business game to train people for collaboration in virtual
enterprises. Online Educa Berlin 2005, 11. International Conference on Technology Supported Learning &Training. 2005.
Wieringa, R.: A Survey of Structured and Object-Oriented Software Specification Methods and Techniques. ACM Computing
Surveys, Vol. 30, N. 4, December 1998.
Windhoff, G.: Planspiele für die verteilte Produktion. Entwicklung und Einsatz von Trainingsmodulen für das aktive Erleben
charakteristischer Arbeitssituationen in arbeitsteiligen, verteilten Produktionssystemen auf Basis der Planspielmethodik.
Dissertation. Bremen, 2001
Zave, P.: Classification of Research Efforts in Requirements Engineering. ACM Computing Surveys, Vol. 29, N.4, 1997.