RUNNING HEAD: Thinking Processes during Garden Design
Unearthing the creative thinking process: fresh insights
from a think aloud study of garden design
Andrew Pringle, Paul T. Sowden
Department of Psychology, University of Surrey, United Kingdom
In press Psychology of Aesthetics, Creativity & the Arts
Author note:
Andrew Pringle, School of Psychology, University of Surrey, Guildford, United
Kingdom and the Insight Centre for Data Analytics, University College Dublin. Paul
Sowden, School of Psychology, University of Surrey, Guildford, United Kingdom.
Both authors contributed to writing this paper with the first draft produced by the first
author based on their doctoral dissertation at the University of Surrey. This research
did not receive any specific grant from funding agencies in the public, commercial, or
not-for-profit sectors. We wish to thank Matthew Peacock for his dedicated work as
the second coder of verbal protocols and his advice and expertise. We also wish to
thank Adrian Banks and Ken Gilhooly for their advice and expertise on the use of
think-aloud protocols and Markov chain models. Correspondence concerning this
article should be addressed to Andrew Pringle, Insight Centre for Data Analytics,
University College Dublin, O’Brien Centre for Science. Belfield, Dublin 4, Dublin,
Ireland. Tel.: +353 017162313. E-mail address: [email protected].
1
RUNNING HEAD: Thinking Processes during Garden Design
Abstract
A number of theories of creativity have converged on the idea that creative thinking
entails shifting between different processes. We attempt to build on recent theoretical
developments through empirical work to examine creativity in the everyday
environment of a garden designer. We asked designers with different levels of
expertise, a matched group of fine artists and a non-designer, non-artist control group
to work on a garden design. We asked them to ‘think aloud’ as they designed and we
recorded audio and video. We coded resultant verbal segments as indicating the
operation of different types of underlying thinking process identified in recent
theoretical work. We then mapped these segments to the video of the designs and
conducted Markov chain analysis to explore how thinking processes shifted as the
design evolved. Finally, we examined the extent to which different types of thinking
process shifts predicted the creativity of the final garden designs as determined by
experts. We found that shifts between associative and analytic thinking processes
predicted design creativity, but only when the operation of these two processes were
tightly coupled in time. The positive association between shifting and creativity was
strongest when analytic thinking processed affective content. These types of shifting
were also elevated at times when a subset of participants switched between working
on different designs; a strategy that positively predicted design creativity. Findings
suggest expansion of mode shifting theories of creative thinking to include the
importance of close coupling between different modes of thinking and of an analytic
mode processing affective content.
Keywords: Mode Shifting, Creative Thinking, Design, Think-Aloud Method, Markov
Chain Models
2
RUNNING HEAD: Thinking Processes during Garden Design
A number of theories of creativity have converged on the idea that creative thinking
entails shifting between different processes (e.g. Basadur, Graen & Green, 1982;
Basadur, 1995; Dietrich, 2004; Finke, Ward & Smith, 1992; Gabora & Ranjan, 2013;
Howard-Jones, 2002; Nijstad, De Dreu, Rietzschel & Baas, 2010). These processes
resemble aspects of broader dual-process theories of cognition (Evans & Stanovich,
2013; Frankish, 2010; Stanovich & Toplak, 2012) and recent reviews have critically
examined this similarity and the implications for our understanding of creativity
(Allen & Thomas, 2011; Sowden, Pringle & Gabora, 2015). Most theories of the
creative thinking process propose that creativity requires the generation of ideas that
are then evaluated and/or honed for their intended purpose, with a growing emphasis
that creativity hinges on the ability to shift between different modes of thinking
supporting generative and evaluative activities (Gabora & Ranjan, 2013; HowardJones, 2002; Kaufman, 2011). In fact Kaufman (2011) has argued, “the highest levels
of creativity require…the flexibility to switch modes of thought throughout the
creative process” (p. 458). Further, computational work has developed shifting
algorithms that model the human creative process (Veloz, Gabora, Eyjolfson & Aerts,
2017) and human cultural evolution (Gabora, Chi & Firouzi, 2013). In addition,
laboratory studies (see Vartanian, 2009; Vartanian, Martindale & Kwiatowksi, 2007;
Dorfman, Martindale, Gassimova & Vartanian, 2008; Beaty, Silvia, Nusbaum, Jauk &
Benedek, 2014) and our own recent psychometric work (Pringle & Sowden, 2017)
have provided empirical support for the positive association between creativity and
mode shifting. However, empirical work has yet to look ‘under the hood’ at an
ecologically valid example of human mode shifting to determine if mode shifting
observed in a real-life creative process is linked to the creativity of the product
produced at its conclusion. Thus, the key goal of the present study was to explore this
3
RUNNING HEAD: Thinking Processes during Garden Design
important issue regarding the ecological validity of ideas about mode shifting. To do
so we examined the creative process in garden design.
Looking ‘under the hood’ at the creative process during garden design
We chose garden design as the everyday environment within which to examine the
creative process for a number of reasons. First, design is a recognized area of creative
endeavor requiring mode shifting to generate and evaluate ideas (Cross, 2011; Dorst
& Cross, 2011). Second, designing a garden is a task that can be engaged in by those
without specialist knowledge but where significant expertise and skill can be
developed with training. This was crucial for the present study as both expert and
non-expert groups of participants were included. Third, professional garden designers
are capable of sketching garden designs in a short time period (e.g. within forty-five
minutes) and often have to do so for clients (Fischer-Tomlin, A, personal
communication, 2013). This was important as the design task had to be short enough
that it could be completed within a single session to make it manageable for
participants and those coding the data.
The data were video footage and verbal protocols generated by a ‘think-aloud’
process as participants worked on designing a garden. Further, the finished creative
product produced at the end of the process; that is the individual’s final sketch of their
garden design, was rated by expert judges for its creativity, design quality and for
how closely it met the design brief. This method closely resembles that used in
previous work in this journal examining creativity in visual art (Fayena-Tawil,
Kozbelt & Sitaras, 2011) and related work in the journal Design Studies that
commonly makes use of the ‘think-aloud’ method and ‘protocol analysis’ to examine
4
RUNNING HEAD: Thinking Processes during Garden Design
the design process (e.g. Atman, Chimka, Bursic & Nachtmann, 1999), which we
elaborate next.
The ‘Think-aloud’ method and ‘Protocol analysis’
The ‘Think-aloud’ method involves participants continually thinking-aloud their
thoughts as they work on a task, in this case designing a garden. In general, the
‘think-aloud’ method is found not to effect task performance (Ericsson & Simon,
1993). It has been used previously to examine components of creativity, namely
divergent thinking (Gilhooly, Fiortou, Anthony & Wynn, 2007) and insight problem
solving (Fleck & Weisberg, 2004), with no differences in task performance found
between groups completing a task while thinking-aloud compared to a control group
completing a task silently. In the present case, all of a participant’s utterances were
transcribed resulting in a verbal protocol for the entire creative process. The visuals
from the video data were recorded alongside the audio of the verbal protocol. This
verbal-visual protocol was then divided into segments, with a segment defined as
words, phrases or sentences of any length that made up one distinct statement about
something such as an idea or topic (Suwa & Tversky, 1997; Atman et al., 1999;
Gilhooly et al., 2007). Short segments (typically 5 to 10 sec’s) were used to allow a
fine-grained analysis of the timing of shifting between modes of thought. Importantly,
a key feature of the present work was its use of a detailed theoretical framework of
mode shifting to inform the coding scheme that was developed as described next.
5
RUNNING HEAD: Thinking Processes during Garden Design
Theoretical framework of mode shifting and coding scheme
A crucial feature of the present study’s approach is that it allows a critical test
between multiple theories of mode shifting at once to determine which best explains
the empirical data. Theories of mode shifting differ with respect to (1) the number of
different components between which shifts occur (2) the role of affective processing
(3) the degree to which the different component processes are coupled, reflected by
how closely together in time they occur (4) whether the frequency of mode shifting is
important and (5) whether the timing of shifts are important.
In general, dual-process theories of creativity and dual-process theories of cognition
include an associative mode of thinking and an analytic mode (Sowden et al., 2015).
Based on this commonality, the decision was made to pool attributes of thinking
processes across models of creativity (Howard-Jones, 2002; Gabora, 2005) and dual
process models of cognition (Evans & Stanovich, 2013; Frankish, 2010; Kaufman,
2011) in order to identify the operation of associative and analytic modes of thinking
in protocol segments.
Further, most models of creativity that incorporate different modes of thinking only
differentiate between the two modes based on their cognitive characteristics (see
Howard-Jones, 2002; Gabora, 2010; Gabora & Ranjan, 2013; Vartanian, 2009).
However, recent neuroimaging work suggests that affective processes, supported by
default and limbic brain regions, are involved in the evaluation of ideas during a
creative task (Ellamil, Dobson, Beeman & Christoff, 2012) and Dietrich (2004) has
proposed a model of the interplay between different modes of thinking in creativity
6
RUNNING HEAD: Thinking Processes during Garden Design
that does emphasize the need to consider affective processing. Thus, the present work
further coded thinking as with and without affective content in the analyses of mode
shifting. Consequently, coding first identified segments as associative or analytic
before assessing whether each segment contained affective content or not resulting in
four overarching codes for coding the verbal protocols: analytic-cognitive, analyticaffective, associative-cognitive, associative-affective.
The final coding scheme, with attributes of the different modes of thinking pooled
across the theoretical models discussed (Dietrich, 2004; Evans & Stanovich, 2013;
Frankish, 2010; Gabora, 2005; Howard-Jones, 2002; Kaufman, 2011) is shown in
table 1 with attributes of each mode of thought shown in the ‘segment code’ column
and the theoretical models that attributes are taken from indicated in the ‘source
model(s)’ column. The ‘explanation’ column explains what each attribute is and the
‘example’ column gives an example of this attribute as it appears within verbal
protocols. This approach allows a comparison of the two component models with the
model that additionally separates out affective processes in analytic thinking derived
from Dietrich (2004).
7
RUNNING HEAD: Thinking Processes during Garden Design
Table 1. Displaying the Coding Scheme used to code segments of visual-verbal protocols with either ‘associative’ or ‘analytic’ as well as indicating the further separation of
segments into those with affective content or not.
Associative mode
Segment code
Source
model(s)
Explanation
Example
Generating ideas/concepts
1, 2, 3
Any new idea or elements of new ideas produced
‘what about a stream here’
Developing, thinking through
1
Building new ideas into previous ideas and
‘and I think the stream and path could both meander
developing existing ideas further
and thicken at the apex’
& exploring ideas
Images, metaphors, analogies
2, 3
Talk concerning visual imagery and use of metaphors
‘the journey through the garden
Linking remote ideas
1, 2
Linking ideas that appear to be disparate
‘a bus makes a journey so I could draw a bus’.
Making associations
1, 2, 4
Making connections between different elements.
‘this is going to be a journey’.
Reasoning based on reference to abstract elements
‘makes me think of drawing into the distance’
Making associations to knowledge and/or prior experiences
‘this reminds me of the landscape architect
(but not evaluating it/them)
George Hargreaves’.
4, 5, 6
Going with gut instinct/intuition/gut feelings
‘I really feel like this should have a wall to it’.
1
Things are coming together
‘a journey suggests a flow from one point to another’.
Memory retrieval
Intuition, instinct,
1
self-evidently valid
Half-baked/
only crudely integrated
but it is not clear how they go together
Insight moment
1, 5
Moment of sudden insight
‘Aha I know what I can do here’.
Spontaneous engagement
3
Playfulness and engagement with fantasy
‘I’m playing with shapes, having ideas which are more fantasy
Associative affective
5
Associative thinking that contains affective content
‘I like curvy lines so I’ll put them in’.
Associative cognitive
5
Associative thinking that only contains cognitive content
‘what about a stream here’
Note. Numbers index the following source models: (1) Gabora & Ranjan, 2013; (2) Howard-Jones, 2002; (3) Kaufman, 2011; (4) Evans & Stanovich, 2013; (5) Dietrich, 2004;
(6) Frankish, 2010.
8
RUNNING HEAD: Thinking Processes during Garden Design
Analytic mode
Segment code
Source
model(s)
Evaluation of design ideas/concepts
1, 2, 4
Explanation
Evaluating ideas, evaluating in the context of something
else
Example
(e.g. design brief, expectations) or
that’s not going to work within the scale’.
‘that’s working/that’s not working,
evaluating with reference to reason
Evaluating remembered experiences/
1, 2
past behaviour
Reasoning justified via logic/evidence
Evaluating remembered info about past design relevant
‘that decision in the past was going against my grain’.
experiences
4
Gives evidence/logical argument behind concrete decisions
‘water is a brilliant way in which to unify a site because
it can go on a journey from top to bottom'
Logical deduction
1, 6
Deduction of causal relationships between elements
‘the scale is x metres so this feature will have to be y metres’.
Fixation
2
Adherence to limited set of ideas/stuck in a rut
‘I’m sort of stuck on this idea really’
with evaluative component
4
Using info from reflection to plan for future
‘this needs further working out, I’d work this out in the future’
Analytic affective
5
Evaluating ideas via affective processing
‘I like/don’t like that’
Analytic cognitive
5
Evaluating ideas via cognitive processing
‘that’s not working’
Planning for future,
Note. Numbers index the following source models: (1) Gabora & Ranjan, 2013; (2) Howard-Jones, 2002; (3) Kaufman, 2011; (4) Evans & Stanovich, 2013; (5) Dietrich, 2004;
(6) Frankish, 2010.
9
RUNNING HEAD: Thinking Processes during Garden Design
In addition to coding segments as reflecting the operation of only associative or
analytic thinking, we also identified segments where the operation of different modes
of thinking cannot be clearly distinguished; that is the segment appears to contain
both associative and analytic processes that are tightly coupled together. We labeled
these segments two modes meshed together to reflect the apparent tight coupling
between associative and analytic modes. In meshed segments, analytic evaluations
such as “I can’t…”, “It’s not going to be…” and “it would be really nice…” are often
expressed prior to the evaluated idea actually being introduced suggesting associativegenerative and analytic evaluation are operating closely together in time. In contrast,
in the verbal protocol segments where attributes reflected the operation of only one
mode with adjacent segments coded for a different mode of thinking there was a clear
distinction in the verbal content reflecting a looser coupling between different modes.
The possibility of tight (meshed) coupling between modes is reflected in Nijstad et
al.’s (2010) cognitive flexibility pathway of their dual-pathway model of creativity,
where the operation of an ‘idea monitor’ (an analytic process) continually checks
generated ideas. In contrast, Gabora & Ranjan’s (2013) model implies that shifting
between modes takes more time, with it necessary to disengage one mode of thought
prior to engaging another, or to shift along a continuum between associative and
analytic to enter a different mode. Thus, the additional two modes meshed code
facilitates analysis of the extent to which the different modes of thinking are closely
coupled together in time allowing us to further explore the fit between our data and
different theoretical models. Further, in keeping with Dietrich (2004) and with Ellamil
et al.’s (2012) observation of the role of affective processing when analyzing creative
ideas, we distinguished two modes meshed segments on the basis of whether they
10
RUNNING HEAD: Thinking Processes during Garden Design
contained affective content or not. Examples of ‘two modes meshed together’
segments from participant’s protocols are shown in table 2.
Table 2. Displaying example segments from participants’ verbal protocols coded as ‘two-modes
meshed’ with two sets distinguished on the basis of whether or not they contain affective content.
Two modes meshed together
Segment
code
No affective
content (i.e.
only cognitive)
Example
'Its not going to be curved, because that doesn't work'
'Oooh, it could couldn't it, you could fold the land slightly'
'I don't think I want the terrace or any presumed terrace straight away next to the house'
Contains
affective
content
'Ah now I saw something really interesting the other day [hidden hedge] and I think that would be
quite fun'
'I really like the idea of this being like a lovely green sort of forest floor underneath this elevated
pool'
'I think that would actually be quite nice that we could actually move the water'
Two final points arising from the theories of mode shifting concern the importance of
the frequency and timing of shifts. Nijstad et al.’s (2010) model of creativity
conceptualizes one pathway to creativity as involving cognitive flexibility in the form
of frequent switching between different categories of ideas and approaches with the
concurrent use of an evaluation mechanism, the ‘idea monitor’; to check the
appropriateness of generated responses. Based on this we hypothesized that creativity
would be positively correlated with the frequency of transitions from associative to
analytic and/or the frequency of two modes meshed segments. Models of creativity
also suggest the importance of the timing of shifts between modes during the creative
process, for example to break out from being “stuck in a rut” or overcoming an
impasse (Gabora & Ranjan, 2013; Howard-Jones, 2002; Sowden et al., 2015). Based
on this we examined mode shifting at time points that may be particularly important
to the creative process of designing a garden, namely at points when participants
11
RUNNING HEAD: Thinking Processes during Garden Design
switched between working on different sketches for a design prior to completing their
final design.
The effects of domain specific expertise
A final important issue was to explore the effect of domain specific knowledge and
skill on the creative process and outcomes. Consequently, we included professional
garden designers, student garden designers, fine artists (with therefore highly
developed drawing skills but without garden design specific knowledge) and a group
of university staff who were neither designers nor artists. The latter group was prescreened for low levels of creative achievement (Low CAQ group), defined as scoring
low (M= 3.58, SD=2.84) on the creative achievement questionnaire (CAQ) (Carson,
Higgins & Peterson, 2005). All participants were given the same task, to produce a
creative design for a garden based on a short design brief. The rationale for including
different groups was to explore if there were expertise related differences in mode
shifting during the creative process and in the creativity of the designs produced as a
product of this process. Professional garden designers were expected to be most
proficient at mode shifting and to produce the most creative designs followed by
student garden designers and fine artists in the middle, with the low creative
achievement group expected to show the least evidence of shifting and the least
creative designs. It was not clear whether students or fine artists would perform
better, with benefits from the student’s greater expertise in garden design possibly
being balanced out by fine artists having greater expertise in drawing skills.
12
RUNNING HEAD: Thinking Processes during Garden Design
Method
Participants
Forty-eight individuals participated. Twelve were professional garden designers (M=
51.72, SD=7.38, 10 females) recruited from the Society of Garden Designers (SGD),
which is the professional body for garden designers in the United Kingdom. All
twelve were registered members of the SGD, which requires that they pass a strict
accreditation process and have been in business for at least three years. Twelve
participants were student garden designers (M= 39.17, SD=17.21, 8 females),
currently studying on garden design courses or who had graduated from courses in the
year prior to the study’s start date. They were also recruited through the SGD and
colleges running garden design courses. Twelve fine artists (M= 53.50, SD=13.42, 9
females), defined as those who had qualifications in fine art and for which fine art
was currently their profession, were recruited from the Royal Society of British artists
(RBA) and Surrey artists websites and from open studios events held in Surrey in the
UK. The twelve low CAQ group (M= 44.10, SD=13.00, 10 females) were members
of non-academic staff who were recruited in person at the University of Surrey and
one language teacher based outside of the University who was recruited through a
personal contact. The study received approval from the University of Surrey Ethics
Committee. Participants were not compensated financially for their time but were
provided with a summary of the study’s findings once all of the data had been
collected and analyzed.
13
RUNNING HEAD: Thinking Processes during Garden Design
Garden design drawing task
The task required participants to produce a design for a garden on A3 paper within a
period of forty-five minutes. Participants were presented with a brief stating that they
should produce a design for a garden ‘based on a journey and the series of
experiences those who walk around the garden will have on this journey’. The brief
emphasized to make the garden as creative as they could but that it should also be
appropriate and work in the context of the brief (the full brief is available as
supplementary material). The brief was devised with assistance from a lecturer of
garden design at a local college and piloted on a fellow PhD student at the University
of Surrey who was studying on a course of garden design, but was not part of the
sample tested here. This helped ensure that the brief was both clear and had validity
as one that a garden designer might work to. Participants were allowed to sketch the
design for the garden in any way they wished (e.g. plan view, in three-dimensions)
and were allowed to produce as many sketches as they wished. They were given
pencils and equipment if needed or allowed to use their own.
Video recording equipment & video analysis software
A digital Sony high-definition video camera was used to video the process of
designing the garden. The video camera was positioned on a tripod focused on the A3
piece of paper and hands of the designer as they sketched their designs. A software
package called Transana (Woods & Fassnacht, 2012) was used to analyze the audio
and video data captured by the video camera. This package enabled segments in the
video to be linked to segments in the verbal reports produced by participants so that
14
RUNNING HEAD: Thinking Processes during Garden Design
both the video and audio data could be used when coding for attributes of different
modes of thinking within the verbal protocol and for other analyses (e.g. if/when
participants switched between different designs, see results section).
Procedure
Participants completed the garden design task individually in their private design/art
studios, in the studios of design colleges or within a design studio set up within the
School of Psychology at the University of Surrey. Eleven of the twelve members of
the low CAQ group and one member of the group of fine artists completed the session
within the studio at the University. All other participants completed the session in
their own studio or the studio of the design college where they were enrolled. The
session lasted a total of one hour and thirty minutes with the garden design task taking
forty-five minutes and the remainder of the time used for participants to read the
information sheet, give informed consent, practice thinking aloud, set up the video
recording equipment and for de-briefing. After providing informed consent
participants were given instructions to help them to ‘think aloud’ as they worked on
the garden design task (the full instructions are available as supplementary material).
Participants were then given two practice tasks to get them used to thinking aloud.
These were to ‘think-aloud’ while they answered the question “what is the sixth letter
after B?” and to ‘think aloud’ while naming ten animals. Following the ‘think-aloud’
practice participants were presented with the brief for the garden design task and
given 45 minutes to work on it. The experimenter was present in the room while
participants completed the task and answered any questions they had during it. Once
participants had completed the garden design task they were de-briefed about the
15
RUNNING HEAD: Thinking Processes during Garden Design
study. The visual-verbal protocols produced by participants were then coded using
the method described next.
Coding of visual-verbal protocols and inter-coder reliability
One coder, the first author of the paper, coded a total of 13,611 segments across 48
participants in the present study using the coding scheme from tables 1 and 2.
Individual attributes were used as a guide to code segments as either showing the
operation of the associative mode, the analytic mode or both (two-modes meshed
together). If after applying the coding scheme it was still not possible to code a
segment with either one of these three modes then the segment was coded as
‘documentation’. Similarly on occasions participants had to be reminded to continue
‘thinking aloud’ or participants asked questions. For both these and ‘documentation’
segments the mode of thinking operating in that segment was coded as ‘unknown
mode’ to reflect that the mode of thinking operating was unclear from its contents.
A second coder, not involved in the research but with expertise in using coding
schemes, coded 205 segments chosen at random from a range of different participants
from different groups and across different time points of participant’s visual-verbal
protocols. Inter-coder reliability was assessed by Cohen’s kappa, calculated on the
205 segments. Simple agreement was found for 80 % of the segments with the kappa
statistic revealing a level of agreement after adjusting for chance of κ= .62, p < .001,
demonstrating substantial agreement (Landis & Koch, 1977). Disagreements
between coders on the coding categories were discussed with the first coder checking
16
RUNNING HEAD: Thinking Processes during Garden Design
through the coding of modes of thinking across all segments to make sure codes were
applied consistently and any disagreements between coders resolved.
The above reliability check only accounts for the modes of thinking coded based on
two-component models of creativity (e.g. Gabora & Ranjan, 2013; Howard-Jones,
2002). A different strategy was used to check for the reliability of the sub-coding of
whether each segment contained affective content or not. First, one coder coded all
protocol segments as containing affective content or no affective content, labeled as
cognitive content. Words in the associative, analytic and two-modes meshed
segments appearing to reflect affective content were identified. These words were
checked against Warriner, Kuperman & Brysbaert’s (2013) database of norms for the
affective meaning of words in order to provide a validation test of the coder’s
subjective judgment. In this database 13,915 words are rated by individuals on a
scale of 1 to 9 on dimensions of valence, arousal and dominance. On each dimension,
higher ratings indicate more positive affect while lower ratings indicate more negative
affect. Using these ratings, it was possible to determine a mean level of affect, on
each dimension, for the ‘average word’ within the database: valence (M=5.06, SD=
1.68), arousal (M= 4.21, SD= 2.30) or dominance (M=5.18, SD=2.16). Segments
coded as having affective content only retained these codes if their content included
words rated at least one standard deviation above or below the mean on at least one
dimension of affect. All other segments were classed as only containing cognitive
content and thus coded as associative or analytic cognitive.
17
RUNNING HEAD: Thinking Processes during Garden Design
Garden Design rating task
Three judges, with expertise in garden design and experience in judging at garden
design shows in the UK, rated the designs using Amabile’s (1996) consensual
assessment technique (CAT) on the dimensions of brief, design and creativity/wow
factor. Brief referred to how well the designs met the requirements of the brief, design
referred to the quality of the design that was evident in design sketches. The
creativity/wow factor was the creativity that judges saw evident in the designs. Judges
were asked to keep the criteria of judgment on different dimensions separate. Designs
were rated relative to one another on each dimension rather than against some
absolute standard for garden design. Ratings were given on each dimension on a 1 to
5 point scale with higher numbers indicating higher scores. Judges were presented
with original copies of all sketches of all designs produced by all participants, blind to
which groups produced which designs. Each judge rated designs in a random order,
defined by the experimenter, and was instructed to make full use of the 1 to 5 point
scale when making ratings. Judges were also instructed to go back and review the
ratings they gave to designs that they rated early in the process once they had rated
many of the designs to help ensure consistency of ratings. Agreement between the
three different judges on their ratings of garden designs was assessed using
Cronbach’s alpha (Cronbach, 1951). This analysis showed a high level of agreement
between judges on all dimensions: creativity (α = .80), design quality (α = .87) and
brief (α = .76).
In light of this good level of agreement, ratings on each dimension
were averaged across judges and used in subsequent analyses.
18
RUNNING HEAD: Thinking Processes during Garden Design
Results
Between-group differences in the creativity and design quality of garden design
sketches
Prior to examining mode shifting during the creative process, it was first necessary to
establish that the products of this process differed across groups on their rated
creativity and design quality. Given the theorized positive relationship between mode
shifting and design quality it was also important to distinguish groups on design
quality. Even if group differences in mode shifting are found, without concomitant
differences in the creativity of the garden design sketches produced, the argument that
mode shifting during the creative process impacted on creative performance would be
undermined.
One student garden designer was excluded from all subsequent analyses after being
revealed to be an outlier with zero transitions from analytic to associative modes (>
three interquartile range’s (IQR’s) from the bottom of the boxplot on this measure of
shifting for the student garden designer group). Ratings on each dimension were then
compared across groups, with the group means and their associated 95% confidence
intervals displayed in figure 1. Creativity ratings and ratings of design quality were
analyzed using analyses of variance. Ratings on brief were not submitted to further
analyses, as there is no theory concerning how mode shifting is related to how well
creative products meet the requirements of the design brief.
19
RUNNING HEAD: Thinking Processes during Garden Design
4
CAT Ratings
3.5
3
2.5
2
1.5
Creativity
Design Quality
1
Brief
0.5
0
Professional
Garden
Designers
Student
Garden
Designers
Fine Artists
Low in
creative
achievement
on CAQ
Group
Figure 1. Displaying mean ratings of the creativity, design quality and adherence to brief for garden
design sketches across each of the four groups. Error bars represent 95% confidence intervals.
A one-way independent ANOVA (Group (4) –professional garden designers, fine
artists and student garden designers, low CAQ group) revealed a significant effect of
group on CAT ratings of the creativity of designs (F (3, 43) = 9.91, p < .001, ηp2=
.41). An identical ANOVA conducted on CAT ratings of the design quality of
designs also revealed a significant effect of group (F (3, 43) = 14.51, p < .001, ηp2 =
.51). Tukey HSD and Games Howell tests were run to examine group differences on
CAT creativity ratings of creativity and design quality respectively. These revealed
the expected advantage for professional garden designers, with their designs rated as
more creative than those of the low CAQ group (p < .001, r = .77) and student garden
designers (p = .04, r = .45) and having a higher design quality compared to all three
other groups: low CAQ group (p < .001, r = .79), fine artists (p = .001, r = .69),
student garden designers (p = .04, r = .51). The only finding involving professional
garden designers not in line with expectations was that their designs were only
marginally significantly more creative than those produced by fine artists (p = .07, r =
20
RUNNING HEAD: Thinking Processes during Garden Design
.47). Against expectations, student garden designers only produced designs that
received CAT ratings for creativity (p = .07, r = .47) and design quality (p = .09, r =
.49) that were marginally significantly higher than those designed by the Low CAQ
group. In line with expectations, designs produced by fine artists were rated as more
creative than those produced by the low CAQ group (p = .03, r = .57) but not higher
on design quality (p = .20, r = .39). As expected, there were no differences in CAT
ratings on either creativity (p = .99, r = .04) or design quality between student garden
designers and fine artists (p = .60, r = .26), suggesting their relative strengths in
design expertise and drawing ability may have balanced each other out.
Having
demonstrated these expected between-group differences in creativity and design
quality, we can now explore whether these differences might be related to the pattern
of mode shifting exhibited by participants in the different groups.
Although the majority of the most creative and highest quality designs were from the
professional garden designers group (seven out of 12 and eight out of 12 respectively)
some of them came from the student designer and fine artist groups. Thus, group
differences don’t tell the whole story; not all professional garden designers produced
the most creative designs with the highest design quality. Consequently, subsequent
analyses on mode shifting will collapse across groups in addition to examining group
differences in order to fully examine the hypothesized link between mode shifting and
product creativity.
21
RUNNING HEAD: Thinking Processes during Garden Design
Between-group differences in verbal output
It was important to compare the mean length of verbal protocols produced by the four
different groups in order to determine if there were any between group differences in
overall verbal output. Differences in overall verbal output could reflect between
group differences in creative self-efficacy (see discussion section for more details).
The mean length of protocols was calculated for each group in terms of both the total
number of segments coded for in protocols and the length of protocols in minutes.
Professional garden designers produced the longest protocols both in terms of
protocol length, M = 50 minutes, 95% CI [47, 53], and total number of segments, M =
339 segments, 95% CI [304, 374], followed by student garden designers, M= 45
minutes, 95% CI [43, 47]; M= 315 segments, 95% CI [265, 365], fine artists, M= 45
minutes, 95% CI [43, 47]; M= 284 segments, 95% CI [233, 335], with the low
creative achievement group producing the shortest verbal protocols, M= 29 minutes,
95% CI [21, 37]; M= 208 segments, 95% CI [134, 263].
A Group (Professional garden designers, Student garden designers, Fine artists, Low
CAQ) ANOVA conducted on length in minutes revealed a significant effect (F (3, 43)
= 15.06, p < .001, ηp2= .51), as did the identical ANOVA conducted on the total
number of segments (F (3, 43) = 6.84, p = .001, ηp2= .32). Post-hoc Tukey tests
revealed that the mean protocol length of the Low CAQ group was significantly
shorter, both in terms of protocol length and the total number of segments, compared
to the groups of professional and student garden designers (p < .01). There were no
other significant between group differences on either measure of protocol length. The
implications of these findings are discussed later (see discussion section).
22
RUNNING HEAD: Thinking Processes during Garden Design
Evidence for the validity of processes identified in think-aloud transcripts
Prior to the main analyses of mode shifting we sought to ascertain evidence that
processes identified from the think-aloud protocols using the coding scheme reflect
genuine underlying thinking processes. In order to do this we obtained a marker of
the quantity of novel ideas from design sketches that participants produced during the
creative process and correlated this with the frequency of segments coded as
‘generating ideas/concepts’ in participant’s verbal protocols. The marker of the
quantity of novel ideas was the number of additional design sketches a participant
produced that showed the addition of novel features compared to previous sketches.
We would expect the frequency of segments coded as ‘generating ideas/concepts’ in
protocols to positively correlate with the number of design sketches with novel
features, given that both are measures of the quantity of novel features generated. We
did indeed find a positive correlation between the frequency of ‘generating
ideas/concepts’ segments and the number of design sketches containing novel features
(rs= .43, p = .001) thus suggesting genuine underlying thinking processes can be
ascertained from the think-aloud protocols.
Analyzing mode shifting using two-component models of shifting
The coded verbal protocols of participants were first examined for the type of mode
shifting proposed in Gabora and Ranjan’s (2013) and Howard-Jones (2002) twocomponent models of creativity; that is shifts between associative and analytic modes
of thinking. Shifts were defined as transitions between adjacent segments in a
23
RUNNING HEAD: Thinking Processes during Garden Design
participant’s verbal protocol, where one segment was coded as associative and the
other coded as analytic. Proficiency in mode shifting was defined here as the
frequency with which a participant transitioned between different adjacent modes of
thinking, from associative to analytic or from analytic to associative, relative to the
frequency with which they did not transition between adjacent modes of thinking, that
is by maintaining an associative (associative to associative) or analytic (analytic to
analytic) mode in adjacent segments. Higher frequencies of transitions between
different modes relative to non-transitions indicate greater shifting proficiency. The
frequency of transitions between adjacent segments where at least one was coded as
‘unknown mode’ was also recorded with these termed ‘unknown transitions’. A
between group (Professional garden designers, Student garden designers, Fine artists,
Low CAQ) ANOVA revealed that there were no systematic differences in unknown
transitions across groups (F (3, 43) = .77, p = .52, ηp2 = .05) and unknown transitions
were also not correlated with either the creativity (rs = - .17, p = .27, N = 47) or
design quality of designs (rs = - .22, p = .14, N = 47). Hence unknown transitions
were excluded from the following Markov chain analyses.
A Markov chain model was used to formally analyze transitions between modes
within a protocol, with modes as categorical events that evolve in a sequence over
time (Kaplan, 2008). An assumption of the model is that the sequence is stochastic,
with the probability of the current categorical event depending only on the categorical
event immediately prior to it (Kaplan, 2008). To illustrate, if events were randomly
distributed then there is a .5 probability that the current mode is associative and a .5
probability that it is analytic. There is a .5 probability that the mode immediately
following the current mode is associative and a .5 probability that the mode
24
RUNNING HEAD: Thinking Processes during Garden Design
immediately following the current mode is analytic. There are thus four possible types
of transition, associative to associative, analytic to analytic, associative to analytic and
analytic to associative. Within the model, the probability of each type of transition
occurring is .25. Transition probabilities thus sum to 1 (Kaplan, 2008). However, in
reality it was expected that the events would not be randomly distributed and that they
would vary between individuals and groups. Thus, taking the example of associative
to analytic transitions:
Transition probability (associative to analytic) = Σ (associative to analytic) / Σ
(associative to analytic + associative to associative)
In words, the transition probability where the mode was associative at time n and
analytic at time n+1 was the ratio of the observed frequency of associative to analytic
transitions out of the total number of transitions in which the start state at time n was
associative. Transition probabilities were calculated separately for shifts between
modes in each direction, from associative to analytic and from analytic to associative
and when the same mode was maintained across consecutive segments: associative to
associative and analytic to analytic. Means for each of the participant groups and
their associated 95% confidence intervals for transition probabilities are shown in
figure 2.
25
Transition Probability
RUNNING HEAD: Thinking Processes during Garden Design
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Professional GD's
Student GD's
Fine Artists
Low CAQ
Analytic to Associative Analytic to Associative
Associative to Analytic
Analytic
to
Associative
Transition Type
Figure 2. Displaying group means for the transition probabilities of the four different types of
transition. Error bars represent 95% confidence intervals.
Analyses of variance performed to examine group differences in analytic to
associative and associative to analytic transition probabilities failed to reveal any
group differences in mode shifting as did a second set of analyses which collapsed
across groups to examine correlations between participants’ scores for the creativity
and design quality of their designs and each type of transition probability. Thus,
when analyzed through the lens of two component models of the creative thinking
process there was no support for the importance of mode shifting during the creative
process of garden design for the creativity and design quality of designs produced at
the culmination of this process.
Analyzing shifts incorporating the notion of an analytic mode with affective
content
In comparison to two component models of mode shifting, we divided segments into
those containing affective content and those without affective content on the basis that
Dietrich’s (2004) model distinguishes affective and cognitive processing in creativity
26
RUNNING HEAD: Thinking Processes during Garden Design
and that, further, Ellamil et al. (2012) have specifically implicated affective processes
in analytic evaluation during the creative process. Thus we hypothesized that the
frequency of transitions between analytic segments containing affective content
(analytic affective) and associative segments containing no affective content
(associative cognitive) would differ between groups and be associated with the
creativity and design quality of the garden designs. For comparison, we also
conducted the same analyses on the frequency of transitions between analytic
segments containing no affective content (analytic cognitive) and associative
cognitive segments.
Although there was no theoretical basis for predicting differences, we did also
conduct analyses on other types of transition (e.g. associative affective to analytic
affective). Unsurprisingly, these were not significant. We do however also have to
account for these different transitions in the Markov chain model hence all possible
transitions from associative cognitive are included in the denominator of the Markov
chain model (see next).
A Markov chain model was again used to analyze transitions between different
components within the present study’s verbal protocol. Taking the example of
transitions between associative segments with cognitive content, labeled associative
cognitive, to analytic segments with affective content, labeled analytic affective:
Transition probability (associative cognitive to analytic affective) = Σ (associative
cognitive to analytic affective) / Σ (associative cognitive to analytic affective +
associative cognitive to analytic cognitive + associative cognitive to associative
affective + associative cognitive to associative cognitive)
27
RUNNING HEAD: Thinking Processes during Garden Design
In words, the transition probability where the segment was associative cognitive at
time n and analytic affective at time n+1 was the ratio of the observed frequency of
associative cognitive to analytic affective transitions out of the total number of
possible transitions in which the start state at time n was associative cognitive.
Transition probabilities were calculated separately for each of the transitions between
associative cognitive and analytic affective and between associative cognitive and
analytic cognitive in each direction giving a total of four transition types. Means for
each of the participant groups and their associated 95% confidence intervals for the
Transition Probability
four types of transition probabilities are shown in figure 3.
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Professional GD's
Student GD's
Fine Artists
Analytic
affective to
Associative
cognitive
Associative
cognitive to
Analytic
affective
Associative
cognitive to
Analytic
cognitive
Analytic
cognitive to
Associative
cognitive
Low CAQ
Transition Type
Figure 3. Displaying group means for the transition probabilities of the four different types of
transition between associative cognitive and analytic affective (on the left) and between associative
cognitive and analytic cognitive (on the right). Error bars represent 95% confidence intervals.
Comparing group differences across the transition types shown in figure 3 suggests
that the only notable difference between groups was between the low creative
achievement group (Low CAQ) and the three other groups in analytic affective to
associative cognitive transitions. Comparing groups separately on each of the four
28
RUNNING HEAD: Thinking Processes during Garden Design
transition types with Tukey’s HSD tests revealed that the group of professional
garden designers demonstrated elevated Analytic affective to Associative cognitive
transition probabilities compared to the low CAQ group (p = .02) as did the student
garden designers (p = .004). All other differences between groups on Analytic
affective to Associative cognitive transitions and on the other three other transition
types shown in figure 3 were non-significant.
In contrast to the analyses conducted on shifting based on a two component model
these analyses suggest some between group differences in the interplay between
different modes of thinking conceived of as shifts from an analytic mode with
affective content to an associative mode without affective content. Findings revealing
an elevation in this type of shifting within the protocols of professional and student
garden designers compared to the low CAQ group mirror those differences found
between the low CAQ group and both designer groups on the rated design quality of
garden designs and between the low CAQ group and professional designers on the
rated creativity of designs. A final set of analyses, collapsing across groups revealed
that transition probabilities for analytic affective to associative cognitive shifting
positively correlated with garden design ratings of the design quality (rs1 = .28, p =
.03, N = 47) but not the creativity (rs = .15, p = .16, N = 47) of garden designs, thus
mirroring the aforementioned between group analyses showing that expertise in
garden design is positively associated with elevated analytic affective to associative
cognitive shifting.
1
Spearman’s rho correlations were run since the variables were not normally distributed.
29
RUNNING HEAD: Thinking Processes during Garden Design
Meshed Modes of thinking coupled together in time
Thus far, analyses have only examined mode shifting conceptualised as shifts
between adjacent verbal protocol segments coded with one mode or the other; termed
shifts in series. However, an alternative possibility is that the two modes may be
tightly coupled, operating closely together in time. To explore the importance of this
‘meshed’ processing the frequency of two modes meshed together segments was
analysed to examine if there were group differences in the operation of different
meshed modes of thinking. Means for each of three types of two modes meshed
together segments and their associated 95% confidence intervals are shown in figure
Two-modes meshed together
segments
4. The type labeled analytic & associative is the measure based on the two14
12
10
8
6
Professional GD's
4
Student GD's
Fine Artists
2
Low CAQ
0
Analytic &
Associative
Associative
cognitive &
Analytic
affective
Associative
cognitive &
Analytic
cognitive
Type of Two-modes meshed together segment
Figure 4. Displaying group means for the frequencies of the three different types of meshed segments
within protocols. Error bars represent 95% confidence intervals.
component model of mode shifting and the associative cognitive & analytic affective
and associative cognitive & analytic cognitive are the measures based on
30
RUNNING HEAD: Thinking Processes during Garden Design
distinguishing analytic thinking with affective content from that without (Dietrich,
2004; Ellamil et al., 2012).
The patterns of means show that professional garden designers exhibited a higher
frequency of all three types of two-modes meshed together segments compared to
other groups but there is large variability on each measure within each group. Oneway analyses of variance (Group (4) -Professional garden designers, Student garden
designers, Fine artists, Low CAQ) were performed to examine group differences on
each meshed measure separately. The ANOVA conducted on associative cognitive &
analytic cognitive two-modes meshed together segments only revealed a marginally
significant effect (F (3, 23.36) = 2.40, p = .09, ηp2 = .14) with Tukey HSD tests again
only showing a marginally significant difference between professional garden
designers and fine artists (p = .07). The remaining ANOVA’s were non-significant.
Given the high level of within group variability in two modes meshed together
measures, additional analyses were conducted collapsing across groups with the
frequency of the three measures of two-modes meshed together correlated with the
creativity and design quality of garden designs. Significant positive correlations were
found between associative cognitive & analytic affective two-modes meshed together
segments and creativity (r = .40, p < .01, N = 47) and design quality (r = .34, p < .05,
N = 47) and between associative cognitive & analytic cognitive two-modes meshed
together segments and design quality (r = .27, p < .05, N = 47) but not creativity (r =
.15, p = .10, N = 47).
31
RUNNING HEAD: Thinking Processes during Garden Design
Mode shifting when participants display flexible behavior
The approach taken to examine mode shifting in the present study allows an
examination of the prediction that there may be specific time points during the
creative process when mode shifting is particularly important (Sowden et al., 2015).
We looked to identify an event in the visual-verbal protocol that could signify such
periods of elevated mode shifting. The event we identified was when participant’s
switched between working on different sketches for a design prior to completing their
final design. Such an event would seem to be underpinned by a period of shifting
between modes of thinking, for example in order to move from generating features of
a current design to judging when it is appropriate to start a new design (Sowden et al.,
2015). Participant’s often produced a number of different designs during their
creative process. Such flexibility has been defined in terms of the tendency to switch
between different approaches (Nijstad et al., 2010). The tendency to produce different
designs on the garden design task seems to reflect switching between different
approaches and would therefore appear to be a measure of flexibility (Plucker, J,
personal communication, 2013). It should be noted here that there is evidence from
the visual-verbal protocols (see supplementary material) that participant’s switched to
working on a new design because they reached an impasse or believed they could
come up with a better idea if they switched to working on a different design on
another sheet of paper. The evidence does not suggest they switched between
working on different designs simply because they finished their first design early.
This is perhaps best exemplified by participant, ID 13, commenting when switching
to work on a different design at five minutes into the design session “ah, you know
could be something better, don’t get stuck on one thing to start with”. Further,
32
RUNNING HEAD: Thinking Processes during Garden Design
participant ID 3 abandoned working on a new design to return to their first design (see
figure 5 and the supplementary materials) suggesting they reached an impasse. Two
measures of flexibility were obtained: (1) a dichotomous measure of whether
participants had worked on the same or different designs from start to finish and (2) a
measure of the total number of different designs participants produced. The criteria
used to define different designs were that they must be wholly distinct, for example a
garden with curves versus a rectilinear garden. Using similar criteria to those used by
Kozbelt (2008), sketches that merely included the addition of some additional novel
features or attempts to make the designs neater were not coded as new designs.
Prior to examining mode shifting during switches between different designs it was
first important to demonstrate that such instances of switching between different
designs were productive; that is they were positively associated with the creativity of
the final garden designs produced. Correlations performed at the level of the whole
sample (N = 47) revealed a significant positive relationship between the total number
of different designs produced and ratings of both creativity (rs = .43, p = .001) and
design quality (r s = .46, p = .001) for final designs. Additionally, grouping
participants into those who had worked on the same design from start to finish (N =
39) versus those who had worked on different designs (N = 8) revealed that more
creative final designs were produced by the group that switched between working on
different designs (M = 3.10, 95% CI = 2.24, 3.97) compared to the group that
remained working on the same design throughout (M = 1.92, 95% CI = 1.65, 2.20, F
(1, 45) = 11.85, p = .001, ηp2 = .21). Similarly, the group that switched between
working on different designs also produced designs with higher design quality (M =
2.83, 95% CI = 1.97, 3.70) compared to those working on the same design throughout
33
RUNNING HEAD: Thinking Processes during Garden Design
(M = 1.55, 95% CI = 1.30, 1.80, U2= 49, p = .002, r = -.44). In sum, these analyses
show that switching between different designs was linked to superior ratings of the
creativity and design quality of designs produced.
The time windows in verbal protocols when participants, who worked on different
designs, switched from working on a current design to starting a new design were
identified. Each time window was examined for evidence of shifts between modes of
thinking previously coded for in the protocol. Evidence of shifting between modes of
thinking within these windows in the verbal protocol would provide support for the
prediction that the timing of shifts is associated with flexibly switching between
different design approaches and thus in turn with the creativity and design quality of
final designs produced.
An example of one instance of switching between different designs by a professional
garden designer is shown in figure 5 for illustration purposes. The full set of instances
of switching between different designs across all participants is shown in the
supplementary materials. Some participants produced more than two designs and each
instance of switching between designs (e.g. design 1 to design 2, design 2 to design 3
etc.) is displayed in a separate diagram, on a separate page in the supplementary
materials. Figure 5 displays a timeline showing the segments from a participant’s
verbal protocol shortly before, during and after they stopped working on one design
and started working on a new design. The timeline starts at the point of the first
utterance of the first verbal protocol segment within the time window. Displayed on
2
A Mann-Whitney test was used to compare groups on design quality since ratings of design quality
were not normally distributed.
34
RUNNING HEAD: Thinking Processes during Garden Design
Figure 5. Displaying one instance of switching between different designs, in this instance performed by
a professional garden designer.
the timeline are time stamps at five-second intervals with increasing time into the
design session/verbal protocol going from left to right. The point at which participants
stop working on one design and the point at which they start work on their next
design are indicated by the arrows on the timeline with the corresponding time stamps
in brackets. Photos of the two designs are displayed above the arrows. In figure 5, the
reason behind labeling the transition from design 1 to design 3 rather than 2 was
because this participant abandoned their second design to return to working and
elaborating on their first design (see also the figure on page 2 of the supplementary
material, which shows design 1 before it was elaborated as well as design 2) after
which they switched to working on a novel third design. The point at which
participants stopped working on a design was defined as the point at which they lifted
35
RUNNING HEAD: Thinking Processes during Garden Design
their pencil from the paper and stopped sketching that design. The point at which they
started working on a design was defined as the point when they put pencil to paper
and began sketching a new design.
Individual segments of the verbal protocol are represented by the colored bars with
the verbal content of each segment shown at the bottom of the figure. The attribute
code or codes given to each segment are shown in the column to the left of the
timeline. The colored bars to the left indicate the relevant modes of thinking. Shifts
between components of thinking are thus indicated by color changes across
consecutive segments. Two-modes meshed together segments are represented by a
pair of different colored bars at the same position in the vertical plane on the timeline
(see figures on pages 11, 19 and 23 of the supplementary material for examples).
The figures in the supplementary material show across the eight participants who
produced different designs there were a total of 24 instances when switches between
different designs were made. On 21 out of the 24 instances when participants
switched designs they evidenced at least one shift between the different thinking
components defined based on the separation out of affective content (Dietrich, 2004;
Ellamil et al., 2012). When the different modes were defined based on the simpler,
two-component model of mode shifting, participants shifted between different modes
on 20 out of the 24 instances when they switched designs.
It was necessary to formally compare the frequency of shifting between modes of
thinking during the periods in the protocol when participants switched between
working on different designs to the frequency of shifting when participants were
working on the same design. In order to do this the frequency of transitions based on
36
RUNNING HEAD: Thinking Processes during Garden Design
the Markov chain model were calculated within a time window when participants
switched between working on different designs. Time windows within verbal
protocols were calculated from 30 seconds downstream of the point when participants
stopped working on one design to 30 seconds upstream of the point when participants
started working on a different design. Thus protocol segments that fell within this
time window were captured. This time window was chosen because it was wide
enough to ensure that all the transitions between different modes of thinking when
participants switched between different designs were captured. Given that previous
work (Ellamil et al., 2012) has emphasized the potential importance of affect in
analytic-evaluative thinking and that this was supported by our earlier analyses, here
we focus on Markov chain analyses that compare shifts between associative cognitive
and analytic affective with those between associative cognitive and analytic cognitive
thinking.
Chi-square tests were run to compare the total frequency of shifting within time
windows to the total frequency of shifting outside of these time windows. The latter
measure was the frequency of shifting displayed by the eight participants who worked
on different designs outside of the time windows summed together with the frequency
of shifting displayed by the remainder of participants (N = 39) who worked on the
same design throughout the garden design task. Within time windows, counts were
produced of the number of each of the two types of transition based on two
component models of mode shifting and transitions based on also coding for analytic
affective content. Transition types that resulted in a count of less than five in any of
the cells were excluded from analysis in order not to violate a key assumption of chisquare (Field, 2009). For the same reason, transitions were collapsed across
37
RUNNING HEAD: Thinking Processes during Garden Design
direction, for example with transitions from analytic affective to associative cognitive
combined with those from associative cognitive to analytic affective. The number of
two modes meshed together segments within time windows was also calculated.
These measures of the frequency of shifting were each summed to produce totals
across all time windows across all of the participants who switched between working
on different designs. Unknown transitions were also included in the analyses because
we wanted to ensure effects reported here remained after accounting for any
differences in unknown transitions within compared to outside of time windows.
A two (within time windows, outside of time windows) by three (shifts between
analytic affective & associative cognitive, shifts between analytic cognitive and
associative cognitive, unknown transitions) chi-square was run to compare the
frequencies of different transition types within and outside time windows. The chisquare revealed that there was a significant difference in the pattern of transition
frequencies when they were obtained within compared to outside of time windows (χ2
(2) = 30.67, p <.001; see table 3 for cell counts).
Table 3. Displaying the observed (count) and expected frequency of the different types of Markov
chain transitions based on Dietrich’s (2004) framework within and outside time windows.
Transition type
Analytic affective-Associative
cognitive
Analytic cognitive- Associative
cognitive
Unknown Transitions
Within time
windows
Count Expected
%
Outside time
windows
Count
Expected
%
27
11
11
516
532
4
54
174
41
203
21
68
1908
9557
1921
9528
16
80
38
RUNNING HEAD: Thinking Processes during Garden Design
Standardized residuals were used to explore the significant chi-square, with
significant differences between observed and expected counts revealed by z-scores
greater than 1.96/-1.96. Within time windows, there was a significantly higher
observed versus expected frequency of shifts between Analytic affective and
Associative cognitive modes (z = 4.7, p < .001) and between Analytic cognitive and
Associative cognitive modes. (z = 2.1, p < .001). It is also important to note that
within time windows there was a significantly lower observed versus expected
frequency of unknown transitions (z = -2.0, p < .01). There were no significant
differences between any observed and expected counts for any transition type outside
time windows.
Table 4. Displaying the observed and expected frequency of two-modes meshed together segments
based on two component models and Dietrich’s (2004) framework within and outside time windows.
Within time
windows
Type of Meshed segment
Associative & Analytic (from two
component model)
Analytic affective & Associative
cognitive
Analytic cognitive & Associative
cognitive
Non meshed segments
Outside time
windows
Count
Expected
%
Count
Expected
%
12
6
5
292
298
2
4
2
2
73
76
1
8
249
4
255
3
95
219
13170
223
13164
2
98
In addition, to exploring the transition frequencies we also assessed the simple
frequencies of the different types of meshed segments. A two (within time windows,
outside of time windows) by two (meshed segments, non-meshed coded segments)
chi-square was run to compare the frequency of two-modes meshed together segments
based on the two-component model of mode shifting (i.e. associative & analytic)
within and outside time windows. The chi-square revealed a significant difference in
the proportion of two-modes meshed together segments versus non two-modes
meshed together segments within compared to outside of time windows (χ2 (1) = 6.97,
39
RUNNING HEAD: Thinking Processes during Garden Design
p = .008; see Table 4 for cell counts). Within time windows, there was a significantly
higher observed versus expected frequency of two-modes meshed together segments
(z = 2.6, p < .001). There were no significant differences in the observed versus
expected frequency of non-meshed segments either within (z = -.4) or outside (z = .1)
time windows.
A two (within time windows, outside of time windows) by three (analytic affective &
associative cognitive meshed segments, analytic cognitive & associative cognitive
meshed segments, non meshed segments) chi-square was run to compare the
frequency of two-modes meshed together segments based on the model that
distinguishes affective from non-affective content within and outside time windows.
The assumption that at least 25 % of cells in the chi-square should have expected
counts of five or more was violated but cells do reach Everitt’s (1977) criterion of
expected values being greater than 1 giving the analysis credibility. The chi-square
revealed a significant difference in the pattern of frequencies of these different types
of segments when they were obtained within compared to outside of time windows (χ2
(2) = 7.83, p = .02; see Table 4 for cell counts). Within time windows, there was a
significantly higher observed versus expected frequency of analytic affective &
associative cognitive meshed segments (z = 2.1, p < .05). Within time windows, the
difference between observed and expected frequencies of meshed analytic cognitive
& associative cognitive segments was not significant (z = 1.80).
40
RUNNING HEAD: Thinking Processes during Garden Design
Discussion
This was the first work to explore the ecological validity of ideas about mode shifting
by examining if mode shifting during the creative process outside of the laboratory is
linked to the creativity of the product produced at its conclusion. Use of the thinkaloud method in conjunction with video data and a detailed protocol analysis of
participants designing a garden in real-time provide a richer insight into mode shifting
than is typically possible using existing laboratory (e.g. Vartanian, 2009; Vartanian et
al., 2007; Dorfman et al., 2008; Beaty et al., 2014) or psychometric (Pringle &
Sowden, 2017) approaches. Importantly, this study allowed a first test of the
hypothesis that proficiency in mode shifting during the creative process is associated
with the creativity of the products, namely final garden designs, produced at the end
of that process. It is necessary to demonstrate this link in order to show the practical
value of research on mode shifting; namely that shifting could impact on the creativity
of the product produced at the end of the creative process.
Results do suggest a link between proficiency in mode shifting during the creative
process of garden design and final design creativity and design quality, but critically,
this relationship hinges on how proficiency in mode shifting is conceptualized. When
proficiency in mode shifting is conceptualized, according to two component models
of creative thinking (Howard-Jones, 2002; Gabora & Ranjan, 2013; Vartanian, 2009)
as the frequency of shifts between adjacent associative and analytic modes during the
garden design process there was no evidence of any relationship between mode
shifting and final design creativity or design quality. When shifting proficiency was
conceptualized according to shifts between Dietrich’s (2004) analytic mode operating
41
RUNNING HEAD: Thinking Processes during Garden Design
on affective content and an associative mode operating on non-affective, cognitive
content then a relationship between mode shifting and final garden design ratings of
design quality emerged. Further, when proficiency in mode shifting was
conceptualized as the frequency of two modes of thinking meshed together operating
closely together in time (Sowden et al., 2015) then a relationship between mode
shifting and garden design ratings of both creativity and design quality emerged.
Results also revealed elevated mode shifting at key time points in the design process
when participants demonstrated flexibility by switching between working on different
designs; a behavior that appears to be underpinned by mode shifting. Importantly,
switching between different designs was also positively associated with the
production of more creative designs with a higher design quality. Thus mode shifting
during these time points may be another index of shifting proficiency linked to
creative performance and design quality.
The majority of models of the interplay between different modes in creative thinking
conceptualize mode shifting as occurring between two distinct components (HowardJones, 2002; Gabora & Ranjan, 2013; Vartanian, 2009). The finding that these twocomponent models of mode shifting are a poor fit to explain the relationship between
shifting proficiency and creativity in the current data suggest the two-component
conceptualization needs to be broadened. Specifically, current findings suggest it
should be expanded in two ways; firstly by introducing the conceptualization that
different modes can act as two modes tightly coupled together in time and secondly
conceptualizing the analytic mode as capable of operating on affective content. Twomodes meshed together reflects verbal protocol segments where it wasn’t possible to
separate idea generation from evaluation, with different modes of thinking appearing
42
RUNNING HEAD: Thinking Processes during Garden Design
to operate in a tightly coupled fashion. Some accounts of creative thinking such as
Njistad et al.’s (2010) dual-pathway model do suggest a close coupling between idea
generation and evaluation, with the conception of an idea monitor that continuously
checks generated ideas.
Importantly, the strongest relationship between the frequency of two-modes meshed
together and garden design creativity occurred when the meshed mode included the
analytic affective as opposed to the analytic cognitive component, together with the
associative cognitive component. The correlation between the meshed measure
including the analytic affective component and ratings of the creativity of the designs
was almost double the size of those of the meshed measure with the analytic cognitive
component. It may be the case that thinking creatively on the garden design task
involved shifts between an associative mode that underpins cognitive idea generation
and an analytic mode that uses affective information, in order to help with the
evaluation of ideas. Functional neuroimaging (fMRI) has been used previously to
examine the brain networks recruited when participants evaluated the quality of
designs for book covers (Ellamil et al., 2012) finding activation during evaluation in
default network regions including the medial prefrontal cortex and posterior cingulate
cortex, areas involved in processing affective information. In fact, Dietrich (2004)
also proposed that the analytic affective component of his model of creative thinking
is underpinned by similar brain areas namely the ventromedial prefrontal cortex and
cingulate cortex, although he has since argued against the notion that creativity can be
localized in the brain (Dietrich, 2015). Other work has shown that affect can have a
direct influence on higher-order cognition (Blanchette & Richards, 2010), supporting
the possibility of the real time influence of affect on judgments, which could help to
43
RUNNING HEAD: Thinking Processes during Garden Design
explain the close coupling between modes reflected in two-modes meshed together
segments. It’s important to recognize that segments coded with the analytic affective
component in verbal protocols may only show that participants are focusing on
affective “gut reactions” to the ideas they have generated. Affective evaluations
could be taking place at other times during the garden design session but not reported
upon. However, a focus on the affective component of analytic processes seems
important in its own right as a means to monitor progress on the creation of garden
designs (Ellamil et al., 2012). Previous work employing think-aloud protocols in the
creative process of visual artists (Fayena-Tawil et al., 2011) and a psychometric
measure of shifting (Pringle & Sowden, 2017) both suggest an important role for
metacognitive processes in creativity. Future work should examine the link between
affective evaluations and metacognitive judgments in the creative process.
The evidence for mode shifting during key time points when participants switched
between different designs is also valuable for expanding our conception of mode
shifting. Specifically, it supports Sowden et al.’s (2015) suggestion that shifting
proficiency could be conceptualized as performing shifts at the right time. When
participants switched from one design to another they appeared to do so because they
had reached an impasse or come upon a better idea and shifting between modes
allowed them to make this evaluation and generate ideas anew for a better, more
creative design. The finding that adopting this strategy resulted in the production of
more creative garden designs with a higher design quality suggests it was effective.
There was indeed elevated mode shifting when participants switched between
working on different designs compared to periods when participants worked on the
same design. The type of mode shifting elevated at these key time points mirrored
44
RUNNING HEAD: Thinking Processes during Garden Design
that found in analyses conducted on average levels of shifting during the entire verbal
protocol, with elevations in instances of two-modes meshed together segments and
shifts between analytic affective and associative cognitive components.
It is important to consider the present findings in light of several limitations. Firstly,
the findings linking mode shifting during the creative process of garden design to the
creativity and design quality of designs only show correlation: they do not
demonstrate that mode shifting impacted on creative performance. Future work could
attempt to entrain or interfere with the types of shifting, for example meshing of
associative cognitive and analytic affective modes, which was associated with
creativity in the current work, to examine if this impacts on creative performance on
the garden design task. Secondly, there was a large amount of noise in the data,
reflecting a high degree of variability together with a large number of unknown
transitions which were also elevated at time points when participants switched
between working on different designs, and thus could be a confound in the chi-square
analysis. This noise in the data could be the result of modes of thinking being
relatively crudely defined. Ultimately, neural markers for the different modes of
thinking should be identified in order to clearly capture each mode and better define
the time points when each occurs (Sowden et al., 2015). The ‘think-aloud’ method has
been used previously with success (Fleck & Weisberg, 2004; Atman et al., 1999;
Fayena-Tawil, et al., 2011) but it still has some potential drawbacks in that concurrent
verbalization may interfere with aspects of designing such as perception during
sketching activity (Lloyd, Lawson & Scott, 1995). Furthermore, professional garden
designers often explain the thinking behind their designs to their clients, which may
present a potential confound as other groups, particularly the fine artists and the low
45
RUNNING HEAD: Thinking Processes during Garden Design
creative achievement group, are much less likely to have experience in verbalizing
their thinking. Similarly, expertise in garden design likely confers greater creative
self-efficacy (Beghetto, Kaufman, & Baxter, 2011) that could in turn boost the verbal
output of experts who are probably more confident reporting their creative thoughts
than those without expertise in garden design. In support of this our findings show
that mean protocol length, both in terms of the total number of think-aloud protocol
segments and the length in minutes of protocols, was shorter for the low creative
achievement group compared to professional and student garden designers. The
failure to account for differences in creative self-efficacy is a limitation of the present
work. A final limitation concerns the variability in the environment in which
participants worked on the garden design task. While the ecological validity of the
present work is enhanced by the fact that fine artists, professional and student garden
designers worked in their own studios, variability in environment was introduced by
the fact that the low creative achievement group did not have personal studios and
instead worked in a studio within the School of Psychology. This variability in
environment could thus have contributed to differences between the low creative
achievement and the other three groups on the creativity and design quality of final
garden designs and mode shifting during the design process.
In conclusion, this work has taken the study of mode shifting from the laboratory to
the everyday context of producing a creative design for a garden. In doing so it has
provided data to support a broader theoretical conception of proficiency in mode
shifting than previous laboratory and psychometric work has allowed for. In
particular, the findings suggest proficiency at using associative and analytic modes of
thinking meshed, operating tightly together in unison, and with an analytic mode of
46
RUNNING HEAD: Thinking Processes during Garden Design
thinking processing affective content is linked to creative performance. It also
provides evidence to suggest proficiency at mode shifting may depend on the timing
of shifts during the creative process. This expanded conception and new measures of
proficient mode shifting should help future work identify links between mode shifting
during the creative process and the creative product, essential to taking the next step
of entraining proficiency in mode shifting to aid people’s creative performance.
References
Allen, A. P., & Thomas, K. E. (2011). A dual process account of creative thinking.
Creativity Research Journal, 23 (2), 109-118.
Amabile, T. M. (1996). Creativity in context. Oxford, UK: Westview Press.
Atman, C. J., Chimka, J. R., Bursic, K. M., & Nachtmann, H. L. (1999). A
comparison of freshman and senior engineering design processes. Design
Studies, 20 (2), 131-152.
Beaty, R. E., Silvia, P. J., Nusbaum, E. C., Jauk, E., & Benedek, M. (2014). The roles
of associative and executive processes in creative cognition. Memory and
Cognition, 42, 1186-1197.
Beghetto, R. A., Kaufman, J. C., & Baxter, J. (2011). Answering the unexpected
questions: Exploring the relationship between students' creative self-efficacy
and teacher ratings of creativity. Psychology of Aesthetics, Creativity and the
Arts, 5 (4), 342-349.
Basadur, M. (1995). Optimal ideation-evaluation ratios. Creativity Research Journal,
8 (1), 63-75.
47
RUNNING HEAD: Thinking Processes during Garden Design
Basadur, M. S., Graen, G., & Green, S. (1982). Training in creative problem solving:
Effects on ideation and problem finding and solving in an industrial research
organization. Organizational Behavior and Human Performance, 30, 41–70.
Blanchette, I., & Richards, A. (2010). The influence of affect on higher level
cognition: A review of research on interpretation, judgement, decision making
and reasoning. Cognition and Emotion, 24 (4), 561-595.
Carson, S. H., Peterson, J. B., & Higgins, D. M. (2005). Reliability, validity, and
factor structure of the creative achievement questionnaire. Creativity Research
Journal, 17 (1), 37-50.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests.
Psychometrika, 16(3), 297-334.
Cross, N. (2011). Design thinking: Understanding how designers think and work. NY:
Berg.
Dietrich, A. (2004). The cognitive neuroscience of creativity. Psychonomic Bulletin &
Review, 11 (6), 1011-1026.
Dietrich, A. (2015). How Creativity Happens in the Brain. Basingstoke, UK: Palgrave
MacMillan.
Dorfman, L., Martindale, C., Gassimova, V., & Vartanian, O. (2008). Creativity and
speed of information processing: A double dissociation involving elementary
versus inhibitory cognitive tasks. Personality and Individual Differences, 44 (6),
1382-1390.
Dorst, K., & Cross, N. (2001). Creativity in the design process: co-evolution of
problem–solution. Design Studies, 22 (5), 425–437.
48
RUNNING HEAD: Thinking Processes during Garden Design
Ellamil, M., Dobson, C., Beeman, M., & Christoff, K. (2012). Evaluative and
generative modes of thought during the creative process. NeuroImage, 59 (2),
1783-1794.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis. Cambridge, MA: MIT Press.
Evans, J. S. B., & Stanovich, K. E. (2013). Dual-process theories of higher cognition
advancing the debate. Perspectives on Psychological Science, 8 (3), 223-241.
Everitt, B. (1977). The analysis of contingency tables. London, UK: Chapman and
Hall.
Fayena-Tawil, F., Kozbelt, A., & Sitaras, L. (2011). Think global, act local: A
protocol analysis comparison of artists' and non-artists' cognitions,
metacognitions, and evaluations while drawing. Psychology of Aesthetics,
Creativity, and the Arts, 5 (2), 135-145.
Finke, R. A., Ward, T. B., & Smith, S. M. (1992). Creative cognition: Theory, research,
and applications. Cambridge, MA: MIT press.
Fleck, J. I., & Weisberg, R. W. (2004). The use of verbal protocols as data: An analysis
of insight in the candle problem. Memory & Cognition, 32 (6), 990-1006.
Field, A. (2009). Discovering statistics with SPSS. London, UK: Sage.
Finke, R. A., Ward, T. B., & Smith, S. M. (1992). Creative Cognition: Theory,
Research and Applications. Cambridge, MA: MIT Press.
Frankish, K. (2010). Dual-process and dual-system theories of reasoning. Philosophy
Compass, 10, 914–926.
Gabora, L. (2005). Creative thought as a non-Darwinian evolutionary process.
Journal of Creative Behavior, 39, 262–283.
49
RUNNING HEAD: Thinking Processes during Garden Design
Gabora, L. (2010). Revenge of the “neurds”: Characterizing creative thought in terms
of the structure and dynamics of memory. Creativity Research Journal, 22 (1), 113.
Gabora, L., Chia, W. W., & Firouzi, H. (2013). A computational model of two
cognitive transitions underlying cultural evolution. In M. Knauff, M. Pauen, N.
Sebanz, & I. Wachsmuth (Eds.), Proceedings of the 35th Annual Meeting of the
Cognitive Science Society (pp. 2344-2349). Austin TX: Cognitive Science
Society.
Gabora, L. & Ranjan, A. (2013). How insight emerges in a distributed, content
addressable memory. In A. Bristol, O. Vartanian & J. C. Kaufman (Eds.), The
neuroscience of creativity (pp. 19-44). NY: Oxford University Press
Gilhooly, K., Fioratou, E., Anthony, S., & Wynn, V. (2007). Divergent thinking:
Strategies and executive involvement in generating novel uses for familiar objects.
British Journal of Psychology, 98 (4), 611-625.
Howard-Jones, P. A. (2002). A dual-state model of creative cognition for supporting
strategies that foster creativity in the classroom. International Journal of
Technology and Design Education, 12 (3), 215-226.
Kaplan, D. (2008). An overview of Markov chain methods for the study of stagesequential developmental processes. Developmental Psychology, 44 (2), 457467.
Kaufman, S. B. (2011). Intelligence and the cognitive unconscious. In R. J. Sternberg
& S. B. Kaufman (Ed’s), The Cambridge Handbook of Intelligence (pp. 442467). Cambridge, UK: Cambridge University Press.
50
RUNNING HEAD: Thinking Processes during Garden Design
Kozbelt, A. (2008). Hierarchical linear modeling of creative artists’ problem solving
behaviors. Journal of Creative Behavior, 42, 181–200.
Landis, R., & Koch, G.G. (1977). The measurement of observer agreement for
categorical data. Biometrics, 33 (1), 159-174.
Lloyd, P., Lawson, B., & Scott, P. (1995). Can concurrent verbalization reveal design
cognition? Design Studies, 16 (2), 237-259.
Nijstad, B. A., De Dreu, C. K. W., Rietzschel, E. F., & Baas, M. (2010). The dual
pathway to creativity model: Creative ideation as a function of flexibility and
persistence. European Review of Social Psychology, 21, 34–77.
Pringle, A., Sowden, P.T. (2017). The Mode Shifting Index (MSI): A new measure of
the creative thinking skill of shifting between associative and analytic thinking.
Thinking Skills & Creativity, 23, 17-28.
Sowden, P. T., Pringle, A., Gabora, L. (2015). The Shifting Sands of Creative
Thinking: Connections to Dual Process Theory. Thinking and Reasoning, 21,
40-60.
Stanovich, K. E., & Toplak, M. E. (2012). Defining features versus incidental
correlates of type 1 and type 2 processing. Mind & Society, 11 (1), 3-13.
Suwa, M., & Tversky, B. (1997). What do architects and students perceive in their
design sketches? A protocol analysis. Design Studies, 18 (4), 385-403.
Vartanian, O. (2009). Variable attention facilitates creative problem solving.
Psychology of Aesthetics, Creativity, and the Arts, 3 (1), 57-59.
Vartanian, O., Martindale, C., & Kwiatkowski, J. (2007). Creative potential, attention,
and speed of information processing. Personality and Individual Differences, 43
(6), 1470-1480.
51
RUNNING HEAD: Thinking Processes during Garden Design
Veloz, T., Gabora, L., Eyjolfson, M., & Aerts, D. (2011). Toward a formal model of
the shifting relationship between concepts and contexts in different modes of
thought. In: D. Song, M. Melucci, I. Frommholz, P. Zhang, L. Wang, & S.
Arafat (eds.), Lecture Notes in Computer Science 7052: Proceedings of the
Fifth International Symposium on Quantum Interaction (pp. 25-34). Berlin:
Springer.
Warriner, A.B., Kuperman, V. & Brysbaert, M. (2013). Norms of valence, arousal,
and dominance for 13,915 English lemmas. Behavior Research Methods, 45 (4),
1191-207.
Woods, D, and Fassnacht, C. (2012). Transana v 2.51. http://www.transana.org.
Madison, WI: The Board of Regents of the University of Wisconsin System.
Footnotes
1. Spearman’s rho correlations were run since the variables were not normally
distributed.
2. A Mann-Whitney test was used to compare groups on design quality since
ratings of design quality were not normally distributed.
52