Giving Ideas an Equal Chance: Inclusion and
Representation in Participatory Design with Children
Janet C Read
Daniel Fitton
Matthew Horton
University of Central Lancashire
Preston, UK
PR1 2HE
+44(0)1772 893285
University of Central Lancashire
Preston, UK
PR1 2HE
+44(0)1772 893277
University of Central Lancashire
Preston, UK
PR1 2HE
+44(0)1772 895151
[email protected]
[email protected]
[email protected]
Participatory design is predated by the ideal of inclusion of children as participants in research that has its roots in sociology.
Hart’s model of participation [13], describes a continuum that
shows the deepening involvement of children as a factor that affects participation in the context of empowerment; thus at the
extremes children are essentially not empowered at all (Rung 1 –
Adult-led activities, in which a youth does as directed without
understanding) or are fully empowered (Rung 8 – Youth-led activities, in which decision making is shared with youth and adults
working as equal partners).
ABSTRACT
Participatory Design (PD) in various guises is a popular approach
with the Interaction Design and Children (IDC) community. In
studying it as a method very little work has considered the fundamentals of participation, namely how children choose to participate and how their ideas are included and represented. This paper
highlights ethical concerns about PD with children within the
context of information needed to consent. In helping children
understand participation in PD, a central aspect is the necessity to
help children understand how their design ideas are used which
itself challenges researchers to seek a fair and equitable process
that is describable and defensible. The TRAck (tracking, representing and acknowledging) Method, is described as an initial
process that could meet this need. This is evaluated, in two forms,
in a PD study with 84 children. The TRAck Method encouraged
careful scrutiny of designs and allowed the researchers to distil
useful design ideas although these were maybe not the most imaginative. There is a trade off between the limitations of applying
such a process to PD against the benefits of ensuring fullinformed involvement of children.
In research settings, when children act as participants, it is common practice to seek consent to gather data and to hold information about the study. This consent assumes that children are
fully aware of the potential, and possibilities, for the data they will
contribute. The extent to which a child can understand what he or
she is consenting to is always a concern when researching with
children. Many refer to children as not being able to give consent
and typically assume that consent for their participation must be
drawn from adults; this view is based on the belief that children
cannot be “fully informed” so cannot give consent [24]. Fill information is, of course, never possible as one can never be sure
where a piece of research will lead or what its effects may be.
Categories and Subject Descriptors
H5.2 [User Interfaces]: Prototyping, User-centred design
When children participate with ideas for design within ICT settings, there might appear to be few concerns about data and the
gathering of consent is often simply about gathering assent to take
part. This is a result of the marrying of concerns about informed
consent to the ‘value’ of the data gathered. Having full information about a study is also about an individual being empowered
to understand what is going on, where the information gathered
may be used and where the results and findings will be used. Inclusion of ideas, and the extent to which each child’s contribution
is collected, valued and used is also an important aspect. To date,
in reported PD sessions, the IDC community, whilst typically
paying attention to reporting the number of children, and their
gender, has not been always clear about how children are informed to consent to participate and how each idea is included.
General Terms
Design, Human Factors, Theory,
Keywords
Participatory Design, Child Computer Interaction, Ethics
1. INTRODUCTION
The Interaction Design and Children (IDC) community considers
Participatory Design (PD) as an accepted, and often preferred,
method for engaging with children in activities where new technologies for children are being developed. Since its inception in
2002, of the 195 long papers published at the annual IDC conference, 35 have included at least some aspects of participatory design (e.g. [16] [31]) (http://www.chici.org/resources). Many other
papers on PD have been published at the annual CHI conference
(e.g.[21], [7]).
This paper adds understanding to this space by presenting an
analysis method (TRAck) for the outcomes of participatory design
sessions that could be described to children so they would understand how their ideas are used and that can therefore provide a
starting point for moving the participation of children in PD sessions from Hart’s Rung 3—where children are consulted but have
limited means to feedback—to Rung 5 where they can know how
and where their inputs go. TRAck is described in two forms and
is considered most useful in large studies where many children
from different places may contribute design ideas as it can be used
to both ensure that all the ideas are considered in an equal way
and it presents a defensible strategy, which could be explained to
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee. Request permissions from [email protected].
IDC’14, June 17–20, 2014, Aarhus, Denmark.
Copyright © 2014 ACM 978-1-4503-2272-0/14/06…$15.00.
http://dx.doi.org/10.1145/2593968.2593986
105
has provoked discussion on the value of one-off participation of
children as opposed to more sustained involvement with the concern being about empowerment [30]. The ethical commentator,
Christensen, [4], voiced concerns about participation highlighting
that although there cannot always be a symmetry of power with
children, there should be great care taken to ensure ethical symmetry as a minimal requirement. Ethical symmetry assumes that
children are treated with the same knowledge and ability to consent as adults would be rather than being considered a special
case. In HCI, the Interaction Design and Children (IDC) community has begun dialogue on some of these themes. The roles of
children as participants in HCI work has been modelled by Druin
et al. [9], and participatory power imbalances were discussed by
Read et al. [26] who delineated participatory design activities with
children along an axis from informant through balanced to facilitated design The ethics of children’s participation in design activities has also been examined by Read et al. [25] who made a case
for children being given full information about the potential use
of, and the funding for, the designs that they contribute.
children, for the reduction from many ideas to a few ideas whilst
ensuring fair representation.
The analysis method, TRAck (tracking, representing and acknowledging) is examined and demonstrated in the context of a
PD case study of the design of a mobile product for teenagers that
encouraged better use of electrical and other energy. The study
involved 84 children aged between 11 and 15 from two different
schools. TRAck was applied firstly as 1 design 1 idea (v1) and
secondly as n designs n ideas (v2) and the results are compared
and discussed. The paper critiques TRAck for its ability to identify ideas for future use and in the context of ensuring the most
ethical and most high-level participation of children in PD work.
2. BACKGROUND
2.1 History and Origins of PD
Participatory design is a broad term that, in HCI, refers to the
involvement of end users as informants in the design of technology. Participatory design has its roots in the participatory approaches to decision making that go back as far as Plato and the
Athenian democracies. The principle was, and still is, that if people were given a voice then a) they would be more likely to be
happy with a decision and b) that the people’s voices had a value.
In terms of methods for participatory design with children, several
meta-methods have been described including co-design [5], design partners [8] and informant design. Each of these approaches
varies in its motivations and focus but the core aspect of all is the
engagement with the user group and the facilitation of design
activities by research teams who deliberately seek to delegate the
design activity and power to the participating user team.
Participatory design emerged in various forms from two different
needs The first, coming mainly from the US, was the need for
technological rationalization where IBM and others sought to
engage with users to ensure efficient design of systems with initiatives like joint application design. The second was the social and
humanist approaches that considered user participation necessary
for collective security and individual autonomy. This latter
movement had support in the UK and Scandinavia with key contributions being socio-technical [23], and contributions from the
Tavistock institution and Norwegian work experiments [29]. Participatory design in HCI has had several pioneers and has been
used in several high profile projects [1], [22],
The effectiveness of participatory design in terms of gathering
ideas and inspirations from children has also been discussed within the IDC community. Here research seeks to explore the idea
generation process [20] or to consider how PD can be used with
specific user groups or specific design challenges [32]. User
groups that have been shown to be able to participate in design
activities include very young children [17] and teenagers [19].
2.3 Ideas, values and ethics
In the IDC literature very few researchers have documented how
they have concerned themselves with the rights and feelings of
children within the context of research using participatory design.
Three studies stand out; one that is concerned with how ideas are
seen to filter through the process, a second that foregrounds the
values of the research team in the PD process and a third that
challenges the research team to be especially honest and candid in
their explanations to the children ahead of a PD event.
2.2 PD with Children
PD with children became fashionable in the late 1980s with influential work by Scaife et al. [28] and Druin et al. [10]. Different
traditions of participation evolved along axes that partially mimic
the two underlying principles, these being, children contributing
to make better technology and children contributing their voices
for empowerment. In the former model, the work on co-design by
Druin [10] has been highly influential. The research lab at Maryland pioneered a co-design practice where small teams of children
worked for several weeks on design projects with adult participants. In this model the children develop skills as designers and
participate in a collaborative incremental process that leads ideas
into a single design brief [35]. An alternative view, which is possibly more in line with what is understood as the Scandinavian
approach, is lauded by Mazzone et al. [18] and others. This practice engages with large numbers of children in short bursts, often
in a single event, and seeks to spread a broad net to capture many
ideas as inspirations to be later used by an adult design team.
Studies of participatory design with children tend to focus on the
philosophical arguments around the involvement of children in the
practice, on the examination of methods to facilitate children’s
involvement, and on the quality of the products and ideas that are
generated during PD.
The first is the work by Guha et al. from 2004, entitled ‘Mixing
Ideas’ [12] .This paper described how the authors adapted codesign methods for use in a single session with a young age group
(aged 4 – 6). The central theme of this paper is that the children
wanted to understand how their ideas were used. Tellingly the
authors write that children were ‘visibly upset’ that their ideas
were not being used and that children were ‘not speaking and not
contributing’ when they perceived that their ideas were not being
listened to. The use of the children’s ideas was the main feature
of this work and the authors proposed a method by which ideas
were mixed up, in small groups, and then mixed again and again
until only one ‘big idea’ remained. The process took place in a
consensus discussion with the participating children. The process
was reported as having sparked imagination and ideas.
The second influential paper is by Iversen et al. in 2012 [16]. This
paper, describing a design session for teenagers, deliberately put
democratic behaviours and democracy at the front of the design
session. In particular the researchers who were facilitating the
design session are reported to have ‘presented their interests’ to
Philosophical debate focuses on the role of the children and on the
extent to which they can participate in meaningful design activities. Early discussion has been outside the IDC community and
has come from the social sciences where participation of children
106
the teenagers ahead of the activity taking place. This action, of
clearly letting the design participants know what the ‘values’ of
the research team were, and the emphasis on the teenage designers
being as informed as possible as to the nature of the task, is the
only evidence found of such practice in participatory design studies with children. This is an example of value-centred design
which explains itself as ‘frontloaded ethics’ [33; 34] and promotes
an early look at the values that are incorporated in design. As
written by Friedman et al. [11], ‘Human values and ethical considerations no longer stand apart from the HCI community but
are fundamentally part of our practice. This shift reflects, at least
in part, the increasing impact and visibility that computer technologies have had on human lives’.
i)
ii)
iii)
iv)
v)
The third work is from 2013 and is by Read et al. [27], in which
the ethics of participation of children is examined. This paper
suggests an approach in which the research team deliberately and
critically examine why they are engaging with children and then
ensure that the children have as much knowledge as possible in
order to decide whether or not to participate. This process, when
applied to design, implies that children should be informed about
how their ideas will be used, where they might gain attribution for
them, and what might happen should their idea ultimately result in
monetary gains. Theoretically this takes a view of as ‘ethical
symmetry’ [4], where children are not considered as any different
to adults in terms of participation. It suggests that in all participatory work due consideration be given to ethical balances and to
informed consent It is generally considered that the more meaningful participation is for children the more beneficial the activity
is across all aspects [30], The IDC community has sought to report methods and approaches that value participation [15].
The method is described in two versions (v1 and v2) – and these
are each then examined in the case study that follows.
3.1.1 Terminology
In the narrative that follows an idea is considered to be something
that could potentially be included in a final design specification
prior to build. One design could have many ideas embedded in it
and a design could be the work of one child, or of a group of children. A design can exist as a single image, or prototype, or could
exist as a series of linked images or prototypes. For the purposes
of discussion a group refers to a collection of individuals whose
designs are linked in so much as they come from a single cohort
of children, e.g. one school class and so merit representation. A
design activity may be repeated with several groups to inform a
single final product specification.
3.1.2 Stage 1 – Setting the Parameters
Having planned a design activity and having carried it out with a
large enough cohort of children to make the merging and tracking
of ideas problematic, the evaluators determine approximately how
many ideas they are aiming to have at the end of the process, (in
the example below, this number is ten) and also the approximate
‘size’ of competing teams (in the example described below this is
between 3 and 5 – ideally these would be very similar or all the
same). The designs from all the participating children are then
placed into teams within the groups, with the number of teams
dictating the number of ideas that will be present at the finish.
Collectively, these three papers spell out a research agenda for the
IDC community suggesting three concerns that need attention.
These concerns can be captured as follows:



There is a need to be able to articulate to children a
method that satisfies them that their ideas are being
considered
There may often be too many children participating
in a design session for there to be a simple amalgam of ideas
The design team are able to agree, in advance, on a
maximum number of key ideas that they are looking to take forward and identify as having come
from the design session
There are available a team of experienced individuals who can examine the children’s designs
Any designs offered are sufficiently mature to convey at least one idea that can be taken forward
From Guha – Ideas need to be Accounted for - IA
From Iversen – Participants need to be Told –PT
From Read – Consent requires Information - IC
The remainder of this paper describes the implementation of a
process that accounts for ideas (IA) in such a way that participants
can be clearly advised (PT) about how their ideas are dealt with in
order that they can give fully informed consent (IC) to participation. The following sections describe this process, show its application in a PD session and then discuss its potential.
3.1.3 Stage 2 – Reporting Candidate Ideas
A set of reporting sheets are developed, one for each team, and
each with as many spaces for candidate ideas, alongside each
design, as there are designs in the team. Thus, if there are three
designs in a team, each design has a space for three candidate
ideas to be recorded, if there are five designs in a team, each has
space for five candidate ideas to be reported (see figure 1).
3. DEMOCRATIC DESIGN PRACTICE
Democracies typically fall into two styles; one is a direct democracy where all participants have an equal and a direct say in decision-making, the second is a representative democracy in which
all eligible citizens contribute by electing representatives who
then carry their voices forward. This second model of democracy
came about by necessity as populations became too large to allow
each member a voice in each instance. The way individuals are
represented is key to democratic action, and is one of the three
practices (deliberation, participation and representation) that denominate democracy [6]. Representation is the creation of proxies
to impersonate the desires of a participating population.
Evaluators are then recruited to inspect the children’s designs (this
is situated in the work from Horn et al. [14] who stress the need
for all ideas to be fairly considered and for consensual opinions).
This group of evaluators individually inspects every design and
lists as many candidate ideas as there is space to write ideas in the
spaces on the reporting sheets. Once all the reporting sheets have
been completed, the activity changes according to whether the 1:1
(v1) or n:n (v2) analysis method is being applied
3.1 TRAck Method
3.1.4 Analysis using TRAck v1 - 1:1
The TRAck Method is described here as a means to ensure representation of ideas and tracking of ideas for PD sessions. A set of
assumptions underpin the descriptions that follow. These are that:
Each evaluator selects one candidate idea from each design to
take forward (in figure 1 this can be seen as a tick in the column
headed individual). These ideas are then referred to as (that single
evaluators) selected ideas. Note that there will always be as many
selected ideas as there are individual designs coming into the process. The different evaluators then come together and consider
107
each others selected ideas choosing from those the most popular
ideas (based on how many evaluators chose that idea as their selected idea) and bring that idea forward as the ‘winner’ for that
team of designs. At this stage there will be as many ‘winners’ as
there are teams, and each winning idea will represent a team
(which in turn will represent a group).
4. CASE STUDY
A participatory design study that took place over four days with
four different groups of children in two different schools is described. The PD session aimed to gather designs for conveying
energy use and was part of a three-year research project. The case
study is written up in three parts, the first describes how the context was set for the pupils, the second describes what happened
during the design session and describes the instruments used, and
the third describes the use of TRAck to analyse the outputs.
4.1 Obstructed Theatre
The first stage in the design journey required the design team to
understand what it was that they wanted from the pupils and to
create a means by which these requirements could be conveyed
without constraining design thinking. It is worth noting that the
design team needed to create a successful usable product and so
foremost in their thinking was the need to gather design ideas.
Probe questions that the design team explored were:
1.
2.
3.
4.
Figure 1- Completed reporting sheet showing how 5 candidate
ideas were transcribed and how 5 individual (1:1) and 5 group
(n:n) were chosen to take forward.
Having answered these questions the team determined to use the
obstructed theatre [28] approach in order to ensure a consistent,
inclusive, informative and open introduction to the design activities. This was considered important as the team were seeking to
repeat the same design session with four different groups. Obstructed theatre is a method based on work by Briggs et al. [3]
who took their inspiration from the use of pastiche scenarios in
design [2]. It has been shown to work with children.
3.1.5 Analysis using TRAck v1 - n:n
In this version instead of the individual evaluator bringing forward one idea from each and every design, each evaluator brings
forward as many ideas from the ‘team’ as there are individual
designs in the team – thus, if there are five designs in the team
then five ideas are brought forward to become the selected ideas.
Theoretically these could all come from one design (whereas in v1
they have to come equally with one from each design). In figure 1
above it can be seen that designs 2 and 4 both contributed two
ideas in this case. After each evaluator has determined his or her
selected ideas (and again there will be as many selected ideas as
there were designs coming into the process) the activity continues
in the same way as in v1 with evaluators coming together and
agreeing ‘winners’ with one winning idea coming from each team.
A script for the obstructed theatre was made as shown in Figure 2.
This script conveyed the key responses to the probe questions:
1.
2.
3.
4.
Both methods ensure that each design is considered and is only
competing with ideas from similar groups and thus makes sure
that each different ‘group’ of children is represented in the final
design discussion space. It could be argued that v1 is more inclusive as every team brings an idea through to the ‘vote off’ but v2
has the advantage of possibly ensuring that should one design
have all the great ideas, these ideas are not lost. Theoretically, v2
should result in better ideas than v1 but can still be considered to
be representative.
4.2 Progressive Design Activities
Designs were gathered over four days in two different schools
with two different year groups in each school. Each instance is
referred to as a design session, thus there were four sessions (A,
B, C and D). These are referenced in Table 2.
Three questions arise from the possible use of this method; these
are;
b)
c)
The product will be a mobile / portable device that informs about energy use and is suited to teenagers.
The uncertainties are the shape, form and affordances of
the product and how it should be interacted with.
The constraints are that it should be unobtrusive.
The key functional requirements are that the product
alerts the owner to excessive energy use, conveys energy used, informs about energy uses of different things,
and communicates with other similar devices.
In accordance with the practice, the theatre script was recorded by
a teenage informant and accompanied with a Flash animation with
stick figures that demonstrated the dramatic action.
3.2 Evaluating the TRAck Method
a)
What is the team aiming to build?
What are the uncertainties of the design space?
What are the constraints that pupils will need to know?
What are the key functional requirements?
4.2.1 Participants
The two schools that participated in the study were standard government funded high schools and were typical of most such
schools in the UK. One was partially sponsored by the Catholic
Church (as are about 10% of all UK state funded schools) and the
other had no religious affiliation. For the benefit of international
readers of this paper it is very common for there to be Catholic
High Schools in UK cities and larger towns – these schools take a
small amount of funding from the Catholic Church and as such
are able to include religious attendance / behaviour as part of their
admissions criteria.
Inclusivity: To what extent does this method ensure that
all the children’s ideas are acknowledged / considered?
Representation: How do the two models (v1, v2), for
representation, differ?
Process - From the point of view of a design team, used
to a more holistic approach, what is the effect of using
this process by which ideas can be tracked?
These questions are examined in the case study that follows.
108
SCRIPT
4.2.2 Apparatus (Progressive Design Activities)
Scene: Two hoodies in the garden – one (A) playing banjo, the
other (B) on his phone
The design sessions were facilitated with the use of a pre printed
A4 (roughly letter sized) design product made up of seven portrait-oriented pages with a small title on each page. It was not
expected that all the pupils would complete all the pages but, having had experience in many school based design sessions, the
research team had constructed this ‘booklet’ in order to ensure
that the participating pupils were kept busy for the whole time that
the study was being carried out (approximately one hour). The
headings on the pages are detailed in Table 1 below and Figure 4
shows an example of three completed pages. At the top of page 1
it also said ‘Use the spaces below to draw your designs’.
Shot: Of upper bodies
{opens to quiet activity – banjo and texting….A suddenly twitches
his leg – in a surprised fashion)
Teen B: what’s up?
Teen A: s’okay – just my Oreo –hang on while I take a look….
(Teen A leans forward – out of vision a touch)
Teen B: Oreo – as in biscuit?
Teen A: (still messing around out of shot) ha – nope it’s summut
me an Dee got given for the energy project… it’s just let me know
my energy use for the last couple of hours…… Organic energy
object I think they call it!
Table 1 - The prompts as written on the design booklet
Teen B: (peering over) it looks weird …. (peering further) what
is it?
Teen A: it’s some sort of ne w computer thing that moves and
wibbles and what not …… (leans forward) see you can squeeze it
and fold it and that………….
Page
Heading
Purpose
1
This is what the device is
To get a view of what the
device looks like and some
of its properties
2
This is how it tells me I’m
doing well
To capture some ideas for
the visualizations of ‘doing
well and doing bad’
3
This is how I check how I
am doing
To gather some ideas about
input and control aspects
4
This is how it looks when I
use it to find out more
information
To capture some ideas
about how the information
aspect might look
5
This is how it tells me I’m
doing badly
To capture some ideas for
the visualizations of ‘doing
well and doing bad’
6
This is how it looks when I
meet my friends and compare how we are all doing
This was intended to capture some connectivity
ideas
7
Anything else…
For any other points of
interest
Teen B: Sound….. what else does it do?
Teen A: well, when I do this it shows me how much gas, lekky
and water I’ve used …….. (moves his arms about) and………
……. I can see what Dee has used as well!
Teen B: cool
(Teen A picks up his banjo again and plays a bit more)
Teen B: you going out with Dee then?
(Teen A doesn’t respond)
Teen B: are you?
Teen A: no!
(Teen A picks up his banjo again and plays a bit more)
Teen B: lookin’ at your doofer seems she uses tons more lekky
than you….
Teen A: yeah…. I think she’s got a Jacuzzi or something..
Teen B: hey we should go try it out …….. do Jacuzzi’s use a lot
of lekky?
(Teen B reaches to his phone as if to go to Google)
4.2.3 Procedure
Teen B: how d’you spell Jacuzzi?
At the start of each design activity, which took place in a scheduled lesson, the research team began by ensuring that the pupils
were aware of the aspects of the work. Pupils were told who was
funding the study and were told that the findings from this design
activity would feed into a research product. They were then told
what that product would (potentially) ultimately be used for.
Without having carried out the TRAck process, it was not possible
at that point to assure the pupils that all their ideas would be used
but it was pointed out that the intention was to examine a fair way
of looking at all the ideas created. Pupils consented verbally to
contribute their design ideas and were also advised that they could
withhold them at the end if they so wished (no one did this). Following that, one of the facilitating team outlined the scope for the
work by playing the obstructed theatre script so no examples were
shown. Pupils were asked to annotate their drawings as appropriate so that the research team could understand them.
Teen A: Jacuz
Teen B: slow down
Teen A: (leaning towards his doofer again) here – it’s on here – I
can put in the hours and the device and………………
Teen B: (moving his phone away and le aning in) that is
something else…
(Teen A carries on banjo playing………)
Teen B: shall we go chippy…
Figure 2 - Obstructed Theatre Script for the design session
As a result these schools typically gather pupils in from a slightly
larger catchment area than secular schools and typically do slightly better (on average) in academic achievement. They also tend to
have a slightly more ethnically diverse pupil base. For this study,
84 (48 boys and 36 girls) children and teenagers from two schools
and two-year groups in the UK contributed towards design ideas.
These were pupils from design and technology and science classes. There was nothing extraordinary or out of the ordinary about
the participating children.
The designs were created collaboratively in self-organized design
groups (DGs) using the pre-printed design booklets. The DGs
were made up of two or three (occasionally four) pupils working
together on a single output. There was no intention to control or
manipulate gender therefore the DGs could be either single or
109
mixed sex. Each session was facilitated by two or three academics from the research team and in each case the teacher was present, but not involved, for most of the session. The facilitators
simply explained aspects of the task and ensured that the pupils
stayed on task. They did not influence the design ideas and did not
comment during the activity on any designs that were created. At
the end of the sessions designs were collected up and marked with
the makeup of the DG (boys and girls) and coded so they could be
associated back to the session from which they originated.
It is worth noting that two of the four (R2 and R3) did v2 then v1
and the other two (R1 and R4) did v1 then v2. Finally the four
evaluators came together and determined the winning ideas from
the selected ideas for, firstly v.1, and then v2.
5. RESULTS
The results are presented in two sections. The first section, gathering designs, describes the variety and the essence of the pupils’
designs and includes data that gives an overview of the ideas generated across each team and reports some ‘success’ parameters of
the PD session. This overview is taken from a synthesis of the
ideas recorded by the four evaluators during the first stages of the
analysis. The second section, evaluators choosing representatives,
considers the results by comparing TRAck v1 and TRAck v2.
4.3 Analysis of Designs
The analysis of the designs took place on a single afternoon in the
university in a large room set aside for that purpose.
4.3.1 Participants
In the examination of the designs, four evaluators each independently examined every design for ideas. These evaluators
comprised one professor of child computer interaction, one lecturer in interaction design, one research assistant in child computer
interaction and one PhD student of human computer interaction.
The evaluators were two males and two females. All four had
experience of working in PD studies. Three of the four had been
present at all or some of the design activities; the fourth had the
procedure, purpose, and context of the design activity explained.
5.1 Gathering Designs
Each DG was able to create some designs for the device and all
pupils completed page 1 of the design sheet. An example for page
1 is given in Figure 3 showing the detail and attention that was
relatively typical in this design activity.
4.3.2 Apparatus
Pre-printed reporting sheets, one for each evaluator, were prepared; for an example see Figure 1. Also available for this activity were the 41 designs that had been created during the design
sessions. These were identified with labels showing the team (1 –
10) and the identifier of the DG within that team (1, 2, 3, 4, or 5).
4.3.3 Procedure
Before any analysis took place, the children’s designs were
grouped in batches keeping the schools and year groups together.
Table 2 shows how these four groups represented each school /
year and shows the number of designs in each group. Each group
was then subdivided so that there were ten teams created. This
aligned to the idea of there being a desire to have ten ideas at the
end of the process but also ensured that the teams were a sensible
size, each comprising 3, 4 or 5 designs with 4 being the norm. In
table 2, the figures in brackets after the team numbers show how
many were in that team (e.g. team 6 had 5 designs, team 3 had 4).
Table 2 - How the designs were grouped for analysis
Year 7 (ages 11–12)
Year 10 (ages14-15)
School 1
A. Teams 1(4), 2(4)
B. Teams 3(4), 4(4),
5(3)
School 2
C. Teams 6(5), 7(5),
8(5)
D. Teams 9((4), 10(3)
Figure 3– Design 6/3 that ended up including one of the ‘winning’ ideas with its stickers.
Most of the designs were adequately annotated but some had to
have some small explanations added at the point of them being
handed in. Figure 3, for example, shows ‘available in many different colours’ that was added in at the close of the session.
As outlined earlier in this paper, the evaluators worked through
the designs in stages. In stage one each evaluator looked at one
design at a time and listed three, four or five (depending on the
size of the team that that design belonged to) candidate design
ideas. Each evaluator began looking at a different team’s designs
but all the teams were evaluated in order; thus evaluator R1
looked at team 1 through to 10, whilst R4 began with team 8 then
did 9 and 10 before working through 1 to 7. Once all the designs
had been looked at in this way the evaluators went through the
teams and selected ideas from their candidate lists to take through
as the best individual (1:1) choices (v1) and as the best group
(n:n) choices (v2) following the process as described in section 3.
5.1.1 Progress through the task
All the teams completed at least the first three pages in the design
booklet. Most of the teams completed five pages although not all
the pages were completed consecutively (Figure 4 shows how one
idea was developed). Several missed page four but went on to
complete page five. There was some evidence to suggest that
some pupils had hastily added content to the latter pages in order
to ‘finish the task’. Table 3 shows the progress through the activities (recorded as number of pages attempted) and the average
110
number of candidate ideas noted by the evaluators for each of the
four groups (A, B, C, D).
5.2.1 Stage 1 – Inclusion in Action
Table 4 shows the performance (shown as percentage of candidate
ideas) of the four evaluators in terms of locating candidate ideas
from the designs during the first stage of the process. The numbers in brackets next to the team number identify how many designs were in that team. The letter after the bracket identifies the
group to which that team belonged.
Table 4 - The performance of locating ideas
Team
R1
R2
R3
R4
1(4)A
75
88
94
56
Figure 4 – Design 5/2 showing how a design could be carried
across the first three pages of the progressive design booklet.
This was also a ‘winner’ with the idea of shaking the device
being carried through to the end.
2(4)A
81
100
100
69
3(4)B
50
75
88
56
4(4)B
81
81
94
94
Note that in Table 3, A and B were from one school, C and D
from another and groups B and D were the older age groups. This
table shows that the three design sessions produced roughly similar outputs; no one session was particularly poor in terms of completion of task or of idea generation.
5(3)B
100
89
100
100
6(5)C
76
72
80
84
7(5)C
76
60
84
56
8(5)C
72
100
92
68
Table 3 The progress made during the four different design
sessions -
9(4)D
81
88
100
69
10(3)D
100
100
100
89
Group
Average Progress
through pages (where 7
implies all pages done))
Percentage of Candidate ideas recorded by
evaluators
mean
79
85
93
74
A
6.5
83
B
5.5
81
C
4.9
77
D
5.2
91
Although not significant, it is noted that the mean percentage of
candidate ideas recorded aligned to the experience of the each
evaluator in participatory design sessions. R3 was the most experienced and R4 the least experienced both at facilitating and interpreting design sessions. R2 was the second most experienced in
the group. The high numbers in this table show that all the evaluators put considerable effort into gathering as many design ideas
as possible from the pupils’ drawings indicating a high level of
inclusion of ideas.
5.1.2 Counting Candidate Design Ideas
The principles of inclusion and representation within TRAck determined that for each design within a team, the evaluators could
potentially list as many candidate ideas as there were designs in
that particular team. Thus in a team with three design groups
(DGs), as in team 10, a maximum of three ideas could be identified for each design. Thus a metric was designed as:
−
= 100
5.2.2 Stage 2 – Representation in Action
In the individual scenario (v1), it is clear that after the evaluator
selected the ideas to go forward then each DG would be represented..
Table 5 - Percentages of Ideas
Team
Representation in TRAck v2
R1
R2
R3
R4
50
75
75
75
where i = the number of ideas identified by the evaluator and n is the number of design teams in the group.
1(4)A
2(4)A
75
100
75
50
This percentage is applied as an average across all evaluators and
all designs within a team in Table 3, and is further applied as a
percentage across a team for an individual evaluator in table 4.
For team 6, shown in figure 1, this percentage would be 76% for
that evaluator as there were 19 candidate ideas listed against an
allowable maximum of 25 (5 design groups in the team so potentially 5 candidate ideas from each = 25).
3(4)B
75
50
50
50
4(4)B
75
75
75
100
5(3)B
66
66
66
66
6(5)C
60
60
60
60
7(5)C
100
80
80
60
5.2 Evaluators Choosing Representatives
8(5)C
80
80
60
80
The TRAck process depends on the ability of the evaluators to
identify ideas (see metric in 5.1.2). With the exception of one
design, in the evaluation by R1, every evaluator was able to identify at least one idea from every design.
111
9(4)D
75
75
75
75
10(3)
D
66
100
100
66
mean
69
76
72
68
Inn the second scenario, (v2) it would
w
be possiblle that representtatiion would be significantly dimiinished, especiallly if all the ideeas
fr
from one team came from one design
d
group. Th
his extreme loss of
representation was
w not evidenceed in the results but clearly som
me
ddesign groups lo
ost their voice with
w this analysiss. Table 5 show
ws
hhow many desig
gns were represented, from eacch team, after th
he
inndividual evaluators made theiir group (v2) seelections. In th
his
taable, for examplle from R1, team
m 1(4)A have 50%
% showing that of
thhe four designs that
t came into th
his consideration
n, only two groups
w
were represented
d when the TRAcck v2 analysis was
w applied.
Tab
ble 7 - 'Winning' Ideas
T
The individual DGs
D that were ‘d
dropped’ (i.e. had
d no contributions
innto TRAck v2) are
a listed in tablee 6.
Group
p
v1 ‘winneer’
v2 ‘winner’
1(4)
Shows greeen and red
(1/1)
Display on a tea shirt
(1/2)
2(4)
A locket ddesign (2/1)
A locket desiign (2/1)
3(4)
The produuct bugs you on
your phonne (3/1)
The product bbugs you on
your phone (33/1)
4(4)
Uses hypnnosis to change
behavior ((4/1)
Uses hypnosiis to change
behavior (4/11)
5(3)
Shake thee device to get
feedback (5/2)
Shake the devvice to get
feedback (5/22)
6(5)
Uses stickkers on things
that want tracking (6/3)
Uses stickerss on things
that want traccking (6/3)
7(5)
The wholle device turns
red or greeen (7/5)
The whole deevice turns
red or green ((7/5)
8(5)
Automatically switches
off other ddevices (8/1)
Can spend pooints earned
from device iin shops
(8/1)
9(4)
*not able to select between fouur tied design
ideas
Energy reducction targets
are shown onn the device
(8/2)
10(3)
*not able to select between twoo tied design
ideas
Looks like a w
watch
(10/1)
Tablee 6 – Teams ideeas that were drropped
R
Rater
Teams dropped (Team and DG sub iden
ntifier)
R
R1
1/3, 1/4
4, 2/4, 3/2, 4/1, 5/3,
5 6/3, 6/5, 8/2,, 9/1, 10/1
R
R2
1/3, 3/2
2, 3/4, 4/2, 5/1, 6/1,
6 6/5, 7/2, 8/5,, 9/4,
R
R3
1/4, 2/3
3, 3/2, 3/4, 4/2, 5/1,
5 6/1, 6/5, 7/5,, 8/2, 8/4, 9/4,
R
R4
1/1, 2/1
1, 2/2, 3/2, 3/3, 5/3, 6/3, 6/5, 7/1,
7 7/4, 8/3, 9//4,
10/2
Ideas from 1/2, 3/1,
3 3/3, 4/3, 4/4
4, 5/2, 6/2, 6/4, 7/3, 8/1, 9/2, 9//3,
110/3 (13 designs) were retained by all of the evaluators and ideeas
fr
from 1/1, 2/1, 2/2
2, 2/3, 2/4, 3/3, 4/1,
4 7/1, 7/2, 7/4
4, 7/5, 8/3, 8/4, 8/5
8
99/1, 10/1 and 10
0/2 (17 designs) were all kept by
y three evaluato
ors
T
This suggested that
t
30 of the designs would still have a goo
od
cchance of being represented
r
in th
he final idea list;; only two designs
w
were rejected by all four evaluato
ors.
By groouping ideas acrross the small tteams, the TRA
Ack Method
ensuredd that each team
m was represented in the final lisst of ideas to
go forw
ward. Figure 5 shows the coonvergence and divergence
(comm
mon features of design processees1) whilst eachh of the 41
design booklets conveeying the particcipants’ ideas w
were studied
and up to 692 potentiall candidate ideas extracted, befoore reducing
them ddown to 41 then further down to 10. This structuured process
attemptts to be highly inclusive in thaat every design bbooklet was
both coonsidered (by 4 evaluators) and actively deconsstructed into
its com
mponents (candiddate design ideaas) before beingg considered
for incllusion.
55.2.3 Stage 3 – Democraccy in Action
T
The four evaluaators agreed two
o sets of ideas to take forwarrd.
A
After this analysiis there were 30 ideas that were the same from v1
v
aand v2 and 11 that
t
were differeent – thus 41 id
deas were broug
ght
fforward across th
he ten teams. Frrom each team th
he evaluators theen
sselected a single ‘winning’ v1 deerived idea and a single ‘winnin
ng’
vv2 derived idea. These ideas are listed in table 5.
55.3 Understtanding TR
RAck
R
Reflecting on th
he evaluation of the TRAck Method,
M
the thrree
qquestions, posed earlier in the paaper, are discusseed here.
55.3.1 Inclusivvity - To whatt extent does this method
eensure that alll the children
n’s ideas are considered
c
aand valued?
Itt is clear that in this process, wh
hich was relativeely lengthy, all th
he
cchildren’s design
ns were at least looked
l
at as eacch design was sy
ysteematically scruttinized. It is po
ossible that som
me ideas were not
n
nnoticed and that some were lost, that will alway
ys be the case an
nd
ccan be attributed
d to either a poo
or articulation off the idea (i.e. th
he
innterpreter canno
ot see what it is about) or by thee evaluator havin
ng
aalready extracted
d 3, 4 or 5 ideass and the TRAcck method not ene
ccouraging deeperr investigation.
Figuree 5 - Convergencce and Divergen
nce within RAM
M
5.3.2 Representatiion - How do the two moddels (v1,
v2) fo r representattion differ?
T
The team that caarried out the stu
udies in this pap
per, having prev
vioously taken a less rigorous apprroach to looking
g at design ideaas,
aagreed that the process of TRA
Ack had them all
a spending mo
ore
tiime and more efffort in consideriing the children’’s work.
As cann be seen in Tabble 7, the seconnd version (v2),, where one
child / one set of childdren could potenntially contribute more than
one ideea to the final ‘‘round’ resultedd in a different set of ideas
going fforward. The ideea of the t-shirt, getting points too spend, and
1
112
httpp://www.designccouncil.org.uk/abbout-design/How
w-designersworkk/The-design-proocess/
the use of a watch would all have been lost given only the one
idea per group. The case of team 8 is especially interesting as
when judged individually (v1), the winning idea from 8/1 was that
the device would ‘switch off other things’ but when multiple ideas
were allowed from one design, a second idea from the same group
‘to get points to spend in shops’ emerged as a winner.
ered when many different children, with different ideas and abilities, are contributing. The TRAck method did generate some
useful ideas and these could be incorporated into a final product.
The Scandinavian approach to PD, the empowerment of children,
and the North American approach, the design of a product are thus
both considered. TRAck provided a mechanism to help children
‘climb the ladder of participation’ by promoting inclusive and
representative participation and will allow children to have more
information in order that their consent could be more meaningful.
In more general terms, the process of ensuring representation in
this way by using the TRAck Method did seem to lose something.
Often the goal of design is to conceive a new and innovative solution and after completing the TRAck process the four evaluators
reflected that they felt some of the most innovative ideas had been
lost. This is particularly interesting as if all 4 evaluators had been
in agreement on which these were they would, of course, have
been the winning ideas. The TRAck Method therefore had a
‘normalizing’ effect whereby the most innovative ideas (as judged
by individuals) may not have been carried forwards but those
judged to have a high level of innovation by all researchers were.
The TRAck Method, as a first attempt to provide for the tracking,
representation and acknowledgement of children as idea generators in PD, enabled divergent and convergent consideration of all
ideas through two democratic processes (v1 and v2) to begin an
investigation of the impact of different forms of inclusion. The
TRAck Method specifically sought to ensure ideas taken away
from design sessions (for inclusion in later stages of a design process) were representative of all groups of participants and could
be traced directly to a child’s contribution. This traceability is
important if, for example, a child wishes to withdraw consent
from participation (and good practice for research governance).
5.3.3 Process - From the point of view of a design
team, used to taking a much more holistic approach,
does this method make any sense?
Through this work some interesting trade-offs were explored,
such as reducing the influence of the designer in order to ensure a
more inclusive participatory design process where the designs that
come through are those from the majority rather than those that
are especially odd, and increasing the number of researchers involved in the design process to ensure a more democratic decision
process.
All members of the research team were able to go through the
process. During the merging and agreement process there was
much debate on what was being valued. It was evident that some
interpretation was being applied but the discussion of this was in
itself valuable and insightful. The team agreed that the TRAck
Method ensured that every design had a chance of being in the
final list of ideas and they also agreed that the method had them
look very hard at some of the more farfetched ideas in order to get
to some understanding.
It is hoped that the documentation of the case study presented here
in its completeness will encourage researchers in IDC to consider
how each stage of a PD process, the initial setting up, the use of
materials, and the tracking of ideas can be reported.
The design ideas that were eventually filtered through were all
useful. Specifically some of these have been incorporated into a
functional product that is now in use. However, at the point of the
evaluation taking place the design team also revisited each of the
41 design booklets to see what had been lost. The team discussed
what would inspire design thinking from the sets of design ideas
they had met. Three designs were highlighted as being particularly inspirational. Note that inspirational in this sense means they
would possibly not be taken literally so for example 4/1 (violence)
would not imply designing violence but the meta-idea of punishment was possibly a theme to take forward.
Further work aims to apply TRAck with more unequal groups
than in the present study and to further study the effects on the
design process. To better understand children’s empowerment,
the research team will examine how the TRAck process can be
easily conveyed to children and will seek to create tools that evaluate this effect on their understanding of participation. Comparative studies using TRAck methods and holistic design ideation
will be considered in order to study what is lost from the design
field when these democratic design processes are employed
4/1 – which comprised a subliminal communicator, a walking
tent, and violent punishment for bad behaviour.
7. ACKNOWLEDGMENTS
9/2 – which incorporated room status controls, room control
states, and always-visible data.
Our thanks to pupils from our two participating schools and their
teachers, thanks also to EPSRC and the UK Funding Councils for
supporting this research with grant EP/I000720/1.
2/3 – a product on the underside of a shoe with a wristband control and a wired UI.
8. REFERENCES
In thinking about ‘lost’ ideas the research team agreed that the
process of listing ideas may have reduced designs to bullet points
rather than allowing holistic thinking. It is interesting to note that
from these three ‘inspiring’ designs, only one of these (9/2) was
retained by all four evaluators (see table 6) but the other two were
retained by three which does suggest some consistent thinking.
[2] Blythe, M.A. and Wright, P.C., 2006. Pastiche scenarios:
Fiction as a resource for user centred design. Interacting with
Computers 18, 5, 1139-1164.
[1] Bjerknes, G., Ehn, P., and Kyng, M., 1987. Computers and
democracy: A Scandinavian challenge Avebury, Aldershot.
[3] Briggs, P. and Oliver, P.L., 2008. Biometric daemons: Authentication via electronic pets. In Proc. CHI 2008 ACM
Press, Florence, Italy, 2423-2432
6. CONCLUSION
The TRAck Method allowed the research team to document how
each child’s ideas had been considered and also allowed representation of ideas from four different groups (two ages and two
schools) into a single design brief. In those regards, TRAck supports a more inclusive and hence a more ethical approach to PD
with children in so far as it makes defensible the process of idea
use and also provides a means by which designs can be consid-
[4] Christensen, P. and Prout, A., 2002. Working with ethical
symmetry in social research with children. Childhood 9, 4,
477 - 497.
[5] Churchman, C.W., 1968. The systems approach. Delaconte
Press, New York.
113
tory design with children. In Proc. CHI 2007 ACM Press,
San Jose, CA, 1371 - 1372.
[6] Davies, G.W.P. and Jegu, P., 1995. Ovide and teledemocracy. Journal of Information Science 21, 5, 383 -389.
[22] Muller, M.J., 2003. Participatory design: The third space in
hci. In The human-computer interaction handbook, A.J. Julie
and S. Andrew Eds. L. Erlbaum Associates Inc., 1051-1068.
[7] Dindler, C., Iversen, O.S., Smith, R., and Veerasawmy, R.,
2010. Participatory design at the museum: Inquiring into
children's everyday engagement in cultural heritage. In Proc.
OZCHI'10, ACM, 1952239, 72-79.
[23] Mumford, E., 1993. The ethics approach. Commun. ACM 36,
6, 82.
[8] Druin, A., 1999. Cooperative inquiry: Developing new technologies for children with children. In CHI99 ACM Press,
592 - 599.
[24] Posch, I. and Fitzpatrick, G., 2012. First steps in the fablab:
Experiences engaging children. In Proc. OZCHI'12 ACM,
497-500.
[9] Druin, A., 2002. The role of children in the design of new
technology. Behaviour and Information Technology 21, 1, 1 25.
[25] Read, J.C. and Fredrikson, M., 2011. What do we take? What
do we keep? What do we tell? Ethical concerns in the design
of inclusive socially connected technology for children. In
Proc. Ethicomp 2011 .
[10] Druin, A., Bederson, B., Boltman, A., Miura, A., KnottsCallaghan, D., and Platt, M., 1999. Children as our technology design partners. In The design of children's technology, A.
Druin Ed. Morgan Kaufmann, San Francisco, CA, 51 - 72.
[11] Friedman, B., 2004. Value sesnitive design. In Berkshire
encyclopedia of human computer interaction, W.S. Bainbridge Ed. Berkshire Publishing Group, Great Barringham.
[26] Read, J.C., Gregory, P., Macfarlane, S.J., Mcmanus, B.,
Gray, P., and Patel, R., 2002. An investigation of participatory design with children - informant, balanced and facilitated design. In IProc. IDC'02 Shaker Publishing, Eindhoven,
53 - 64.
[12] Guha, M.L., Druin, A., Chipman, G., A, F.J., Simms, S., and
Farber, A., 2004. Mixing ideas: A new technique for working
with young children as design partners. In Proc. IDC 2004
ACM Press, College Park. Maryland, 35 - 42.
[27] Read, J.C., Horton, M., Sim, G., Gregory, P., Fitton, D., and
Cassidy, B., 2013. Check: A tool to inform and encourage
ethical practice in participatory design with children. In Proc.
CHI2014 ACM, 2468391, 187-192..
[13] Hart, R., 1997. Children's participation: The theory and
practice of involving young citizens in community development and environmental care. UNICEF.
[28] Read, J.C., Fitton, D., and Mazzone, E., 2010. Using obstructed theatre with child designers to convey requirements.
In Proc. CHI2010 ACM Press, Atlanta, GA.
[14] Horn, D. and Salvendy, G., 2006. Consumer-based assessment of product creativity: A review and reappraisal. Human
Factors and Ergonomics in Manufacturing 16, 2, 155 - 175.
[29] Schuler, D. and Namioka, A., 1993. Participatory design:
Principles and practices Lawrence Erlbaum, Hillsdale, NJ.
[30] Sinclair, R., 2004. Participation in practice: Making it meaningful, effective and sustainable. Children and Society 18,
106 - 118.
[15] Horton, M., Read, J.C., Mazzone, E., Sim, G., and Fitton, D.,
2012. School friendly participatory research activities with
children. In Proc. CHI2012, ACM Press.
[16] Iversen, O.S. and Smith, R.S., 2012. Scandinavian participatory design: Dialogic curation with teenagers. In Proc.
IDC'12, ACM, 106-115.
[31] Thang, B., Sluis-Thiescheffer, W., Bekker, T., Eggen, B.,
Vermeeren, A., and De Ridder, H., 2008. Comparing the creativity of children's design solutions based on expert assessment. In Proc. IDC 2008 ACM Press, Chicago.IL, 266 - 273.
[17] Marco, J., Cerezo, E., Baldasarri, S., Mazzone, E., and Read,
J.C., 2009. User-oriented design and tangible interaction for
kindergarten children. . In Proc. IDC'09, ACM Press, Como,
Italy, 190 - 193.
[32] Theng, Y.L., Nasir, N.M., Thimbleby, H., Buchanan, G.,
Jones, M., Bainbridge, D., and Cassidy, N., 2000. Children
as design partners and testers for a children's digital library.
In Proc. ECDL2000 Springer Verlag, 249 - 253.
[18] Mazzone, E., Ivari, N., Tikkanen, R., Read, J.C., and Beale,
R., 2010. Considering context, content, management, and
engagement in design activities with children. In Proc.
IDC2010 ACM Press, Barcelona, 108 - 117.
[33] Van Den Hoven, J., 2005. Design for values and values for
design. Information age, 7 (2) 4 - 7.
[34] Van Den Hoven, J., 2008. Moral methodology and information technology. In The handbook of information and
computer ethics, K.E. Himma and H.T. Tavani Eds. Wiley,
New York, 49 - 69.
[19] Mazzone, E., Read, J.C., and Beale, R., 2008. Design with
and for disaffected teenagers. In Proc. Nordichi 2008 ACM
Press, Lund, Sweden 290-297.
[35] Yip, J., Clegg, T., Bonsignore, E., Gelderblom, H., Rhodes,
E., and Druin, A., 2013. Brownies or bags-of-stuff? Domain
expertise in cooperative inquiry with children. In Proc.
IDC2013. ACM Press, 201 - 210.
[20] Mazzone, E., Read, J.C., and Beale, R., 2008. Understanding
children's contributions during informant design. In Proc.
BCS-HCI'08 BCS, 61 - 64.
[21] Moraveji, N., Li, J., Ding, J., O'kelley, P., and Woolf, S.,
2007. Comicboarding: Using comics as proxies for participa-
114