Ethical issues in developing a game-based virtual training system for final assembly lines in
automotive manufacturers
Setia Hermawati, Nastaran Dadashi, Glyn Lawson, John Wilson
Human Factors Research Group
The University of Nottingham
Abstract
This case study reports on ethical issues at various development stages of a game-based virtual
training system for automotive manufacturers in a multidisciplinary and multinational research
consortium. A report that anticipated potential ethical issues, created within three months of the
research project commencement, was reviewed and compared to the actual ethical issues that arose
as the research project progressed. The review and comparison revealed that, similar to a design
process, ethical issues should be addressed iteratively to match the dynamic nature of collaboration
within the consortium.
Key words: game-based virtual training, enterprise data, training data, automotive manufacturers
1. Technologies involved and background to the case
Virtual training is needed in automotive manufacturers because hardware-based training is costly
and takes time to prepare; resulting in a situation where assembly operators have to cope with
learning new assembly steps at final assembly lines, causing unnecessary rework due to assembly
faults. Motivating by this, a multidisciplinary and multinational research consortium was formed to
develop a virtual training system. The three years research, commencing in 2011 and ending in
2014, is funded by European Union Seventh Framework Programme for Research (FP7). It involves 7
partners, consisting of two automotive manufacturers, four research and education organisations
and a software developer SME.
The virtual training system teaches assembly steps – i.e. which part goes where and when, which
tools to use and what quality check is required – using the latest gaming technology that allows
interaction through gestures (Gorecky et al., 2012). It allows final assembly line operators to learn
assembly steps interactively at a very early stage, even before the physical mock-up or training
hardware is ready. In doing so, the training system optimises the use of existing enterprise data by
automatically synthesising assembly information and CAD data; drastically reducing the effort and
time for setting-up virtual training content which currently is the biggest drawback of virtual
training.
The virtual training system consists of three components (Stork et al., 2012). Figure 1 illustrates the
relationship between these components; details of each component are explained below:
1. A central information hub. This component has two functions: i) performing data mining,
synthesising and storing the semantic relationship between assembly information and CAD
data; and ii) storing training data and best practices of trainee assembly operators.
2. A training simulator. This component represents an interactive virtual assembly simulation
which is used to train trainee assembly operators. This component also captures expertise
of experienced assembly operators by allowing them to share best practices in performing
assembly operations. Gesture recognition technology, such as Kinect and Wii, are used to
support user interaction with the game-based virtual training.
3. A training-enterprise data management platform. This component allows users to create
and review training as well as modify and update enterprise data.
Figure 1. The components of the training system
2. Ethical issues
The development of the training system is only feasible through a collaboration of multidisciplinary
partners - with each partner having their own contribution, interest and a future plan on the uptake
of the training system. Although the two automotive industrial partners in the consortium are not
directly competing for the same market, on the success creation of the virtual training system, the
consortium needs to consider the availability of the tool and information it contains to potential
competitor of the two automotive industrial partners. Furthermore, access to real enterprise data
and actual end-users/relevant stakeholders of automotive industrial partners are required to fulfill
the ambition of producing a fully functioning prototype as a proof of a concept.
Acknowledging the need to review and address the ethical issues related to the above, the
consortium agreed to produce a report that specifically addressed potential ethical issues. This
report was submitted as a project deliverable within three months of the research project
commencement. In the report, titled “Informed Consent and Evaluation Process” and produced by
human factors researchers of The University of Nottingham, potential ethical issues at the
development and evaluation-dissemination stages of the project were identified and measures to
resolve them were recommended. The key actors potentially affected by the ethical issues are
consortium members, trainee assembly operators, other key stakeholders in the automotive
manufacturers and third party organisations or individuals. Table 1 shows the breakdown of
responsibilities for each key actor. This case study provides a reflection on how the ethical issues
that were identified early in the project evolved as the development work progressed.
Table 1. Responsibilities breakdown for each key actor
Why are they responsible?
Consortium
members
Trainee
assembly
operators
Other key
stakeholders
Third party
organisations
or individuals
Have access to enterprise
data & make decisions on
the design of the training
system
Use the system to train,
indirectly allowing their
personal & training data to
be recorded
Either use, have access to
the training system or
indirectly affected by the
deployment of the training
system
Possibility to harvest
unauthorized enterprise
data
What
are
responsible for?
they
How to identify the object of
responsibility & discharge the
responsibility?
Security of enterprise
data, access of personal
& training data
Understanding what
data will be kept & how
it will be used
Understanding the
sensitive nature of the
data held on the training
system & the limit of
what they can do with it
Through the “Informed
Consent and Evaluation
Process” report which
anticipated potential ethical
issues & provided
recommendation on how to
address them
N/A
2.1 Development stage
The deliverable report identified four potential ethical issues that were anticipated to occur during
the development stage. Each of the issues is discussed below:
1. Data collection. The deliverable report recognised that ethical issues could arise from data
collection through a user study whereby relevant stakeholders from the automotive
manufacturers were required to disclose detailed information of the working processes of
their organisations as well their personal views. Therefore, the deliverable report
recommended the use of information sheet and consent form for participants of a user
study. These suggestions were subsequently adopted in the user study. The user study
(Hermawati and Lawson, 2013) was mainly in the form of semi-structured interviews, with
more than one interviewee at a time. Due to the sensitivity of the disclosed information, no
audio or voice recording was made and interviewees’ responses were recorded in written
forms which were then stored in accordance with the research and education organisation’s
guidance on ethical conduct.
The semi-structured interview was led by human factors researchers and supported by endusers representatives who are employees of the automotive manufacturers and members of
the consortium. The presence of the end-users representatives was required as the human
factors researchers did not speak the native language of automotive manufacturers’ relevant
stakeholders.
Furthermore, the end-user representatives’ familiarity with internal
organisation jargon was certainly more helpful than involving an outsider as a translator.
However, the presence of an end-user representative might have made the participants
wary when expressing their candid views, despite guarantees on data anonymity. This
circumstance was overlooked in the deliverable report. This was likely due to the fact that
the report was created three months prior to the commencement of the user study.
2. Data storage. Due to the distributed nature of the project and the call for collaboration
between various partners, the deliverable report also identified the need for a safe and
reliable means for storing the data with different levels of access authorities. This issue was
resolved by adhering to the consortium members’ policy related to data storage. In addition
to this, access to research project data was strictly limited to individuals who were directly
involved in the development of the training system and was managed directly by the
consortium project coordinator.
3. The protection of various parties’ data. This ethical issue was considered and agreed prior to
the launch of the research project in the form of a consortium agreement. The consortium
agreed upon the terms and requirements of publication policy in order to avoid any
compromise in the commercial interests of the involved parties. Similarly, written consent
from the relevant parties was also required in using names, logos and trademarks. Moreover
issues associated with the protocols of an ethical and effective cooperation amongst various
parties were listed and agreed within the consortium. However, even with the support of
the agreement, the sensitive nature of the organizational and product information of
automotive manufacturers proved to be challenging. On several occasions, additional
discussions were required to decide on the level of details of the user study results (due to
sensitive organisation information) that could be published on the internal consortium
website.
4. Knowledge management. Knowledge management involves asking people – experienced
assembly operators – to “share” their knowledge and capture them via the training system.
This can be inhibited by a fear that by sharing their expertise with other employees the job
security of the knowledge sharer might be put at risk. The deliverable report emphasised
the need to explain what the training system would do in utilising any shared knowledge.
This recommendation was appropriately followed and communicated during the semistructured interview. Surprisingly, as shown in the quotation below, the interview revealed
that experienced operators were willing to share their expertise with other operators or
even engineers.
“Would be better to have people who can be more flexible to work in different groups..”
“… More people who can do the same job. When one colleague is ill, there is only one
other...”
Two additional unforeseen ethical issues were also encountered during the actual development
stage. The first ethical issue was related to the training data and history of trainee assembly
workers. The issues revolved around who should be able to access these data and how much data
should be made available. There was a deep concern from assembly operators that training data
and history data might be linked to performance appraisal for trainee assembly workers. This was
despite the fact that the storage of training date was motivated by the intention to monitor and
inform trainers on the learning progress of trainee assembly operators. The second ethical issue was
related to the game-based nature of the virtual training system, particularly concerning inducing
motivation and user experience through competition between players. There was a concern that
allowing trainees to compare their training progress among each other might foster unhealthy
competition within a team. The aforementioned issues were successfully resolved by establishing
discussion with trade union of automotive manufacturers which provided a platform to inform the
design of the training system whilst at the same time seeking their opinion and ensuring that their
concerns were addressed.
2.2 Evaluation and dissemination stage
The consortium plans to undertake two stages of evaluation (formative and summative) at both of
the automotive manufacturer partners; these require direct involvement of end-users to investigate
usability issues and the effectiveness of the virtual training system. Furthermore, a push towards
bigger impact dissemination is also planned once the first prototype of the system is created.
Relevant ethical issues which are identified by the deliverable report for this stage are:
1. Informed consent and anonymity of evaluation trial studies. Informed consent relates to
providing sufficient information about the research project so that participants can freely
decide about their involvement. Sets of informed consent agreement forms used previously
by the University of Nottingham were recommended for use and were communicated to the
consortium members. Another issue that was identified in the deliverable report was
whether or not results from any of the trials would be made anonymous. As individuals from
the general workforce would be asked to participate in evaluation trials, protocols to ensure
confidentiality of results were recommended to be established, such that no individuals
could be recognized either directly or indirectly via a description of their role etc. This
recommendation will be taken into account in the evaluation trial which commences in a
couple of months.
2. Health issues associated with the use of particular technologies. The type of technology that
would be used in the training system had not been decided during the writing of the
deliverable report. As such, it only provided general guidance related to possible health
issues and exclusion of participants due to use of certain virtual technology e.g. virtual
reality sickness due to the utilisation of head-mounted display. The deliverable report
emphasised the need of informing participants the potential risks from the use of such
technology.
In addition to adhering to the recommendation given, this research project also took
preventive measures. The human factors researchers identified type of injuries that were
associated with gesture recognition technology and communicated them to the member of
consortium who was responsible for the system development, such that preventive
measures could be integrated early in the development.
There were also additional unforeseen ethical issues related to the exploitation of the research
project outcome. In the annual review of the project progress, the consortium was urged to form a
more detail and concrete exploitation result. Some issues that were recommended by the reviewer
panel to be addressed included detailed definition of project outcomes that could be exploited,
tentative owners in the consortium for these outcomes, and claims on background knowledge of
foreground activities for the result development. It was also recommended that for each result,
each partner should point out what they intend to do with it i.e. whether it would be used to make
something internally, or to make something that would be sold or licensed. The lengthy request for
details suggested that ethical issues related to intellectual property could become quite complex for
a multidisciplinary project. Another unforeseen ethical issue was the confidentiality of enterprise
data related to showcasing the virtual training system to a wider public. The use of actual enterprise
data proved that, despite the complexity of enterprise data, merging assembly information and CAD
data was indeed achievable. However, this also resulted in the risk of exposing enterprise data
indirectly to third parties while showcasing the virtual training system.
3. Lessons learned
It is evident that simply producing a deliverable report on ethical issues at the beginning of a
research project commencement is clearly not sufficient. As the development work advances and
decision regarding research activities are made, the ethical issues that are encountered by the
consortium also changes dynamically. Therefore, although it is possible to foresee potential arising
ethical issues, predicting the level of details and breadth of measures which need to be taken to
address them can be challenging. Thus, similar to a design process ethical issues should be
addressed in an iterative manner.
The consortium also learned the following lessons related to ethical issues from the multidisciplinary
and multinational research project:
- The inclusion of end-user representatives from an organisation during the user study
benefits the project as it allows verification of organisation information quickly as well as
increasing the sense of ownership. However, their presence during the user study might
cause internal conflict for the invited key stakeholders in putting forward their candid views.
In the case where interviewers and interviewees do not speak the same language, the
decision to involve end-user representatives or an external individual to assist the interview
should be based on the sensitivity of the information that is sought after e.g. whether or not
the disclosed information could jeopardize interviewees’ position within the organisation.
Permission should be sought from the organisation prior to engaging an external individual
as a translator.
- Multidisciplinary research group means that it is likely each partner will bring their own
expertise and past research to be implemented as part of the research project. As was
found in this project, issues related to Intellectual Property and further exploitation of the
results could be quite complicated and, as such, sufficient details are required. This can be
complicated further if the results can be broken down into smaller components that could
subsequently be exploited.
- Incorporating training data (training history and performance) is not as straightforward as it
may seem. Training data is normally handled by Human Resources and may be used as part
of their assessment for individual performance. However, in this research project, Human
Resources personnel were not directly involved as the aim of collecting the training data was
to aid the learning process of trainees. Despite the clarity of this aim, concerns related the
use of data, who has access to the data, how long the data should be stored, etc. still need
to be addressed.
- In contrast to the personal and study data, adhering to the ethical issues regarding
enterprise data is more challenging. Data storage issues aside, the level and extent that
enterprise data can be divulged to a wider public is not clear cut and is not limited simply to
data anonymity to avoid recognition of data at personal level. Although this issue can easily
be resolved by conferring with the organisations involved, additional lead time will be
required as each organisation has their own mechanism to review publicity regarding their
enterprise data.
4. Attributions
The authors would like to acknowledge the contribution to this work from all of the members of the
Virtual Simulation and Training of Assembly and service processes in digital factories - VISTRA project
(ICT-285176). This project (http://www.vistra-project.eu) is funded by the European Commission’s
7th Framework programme.
5. References
Stork, A., Gorecky, D., Michel, F. (2012) Enabling virtual assembly training in and beyond the
automotive industry.
Gorecky, D., Lawson, G., Mura, K., Hermawati, S., Overby, M. L., 2012. User-centred design of a
game-based, virtual training system. In: Trzcielinski, S. and Karwowski, W. (Eds.) Advances in
Ergonomics in Manufacturing.
Hermawati, S. and Lawson, G., 2013. User-centred design of virtual training for automotive
industries. In Proceedings of International Ergonomics and Human Factors Conference 2013,
Cambridge.