Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
Supporting Creativity with Powerful Composition Tools
for Artifacts and Performances
Ben Shneiderman
Department of Computer Science, Human-Computer Interaction Laboratory
Institute for Advanced Computer Studies, and Institute for Systems Research
University of Maryland, College Park, Maryland 20742
[email protected]
Abstract
Modern software such as word processors, slide
preparation/presentation tools, or music composition
packages are designed to produce artifacts or
performances. Now, some designers are expanding their
goals to include creativity support into their software.
This essay builds on the genex framework for creativity,
which has four phases and eight activities. It focuses on
the composition activity by considering tools to support
initiation, revision, and evaluation. Some existing tools
provide partial support that suggests ways in which to
develop more ambitious low, middle, and high level tools.
The goal is to enable more people to be more creative
more of the time.
1. Introduction
Software tools have enabled users to accomplish
increasingly
ambitious
financial,
manufacturing,
information retrieval and other tasks. The movement from
routine to more innovative tasks is leading some designers
to re-examine their software and enhance support for
creativity. Initial efforts focus on ordinary tasks such as
web-page layout or document design, but some designers
are attempting to provide more powerful support for
artistic and scientific endeavors. While paradigm shifting
breakthroughs such as those made by Picasso or Einstein
may not be facilitated by current tools, it does seem
possible to enable more people to be more creative more of
the time.
Creativity has long been seen as a mysterious process
[3]. However, a growing consensus about the phases of
creativity is beginning to enable software designers to
develop powerful tools [5, 6, 7]. This consensus inspired a
four-phase framework called genex (generator of
excellence) that was formulated with the intention that it
be conveniently implemented in software [15]. The genex
framework combined the creativity models that focused on
inspirational moments with those that suggested structured
processes and those that considered a broader social
context. The four phases of genex are:
- Collect: learn from previous works stored in
digital libraries, the web, etc.
- Relate: consult with peers and mentors at early,
middle and late stages
- Create: explore, discover, compose, evaluate
possible solutions
- Donate: disseminate the results and contribute to the
digital libraries, the web, etc.
Users may move through the phases in order, but often
they return to collect further information, ask for help and
emotional support from trusted confidants, and keep trying
alternative solutions. When they find a promising result,
there may still be much effort to refine it, make it viable,
and disseminate it to appropriate people. This four-phase
framework led to a research agenda with eight interrelated
activities that could benefit from software support [16],
Figure 1:
- Searching and browsing digital libraries
- Consulting with peers and mentors
- Visualizing data and processes
- Thinking by free associations
- Exploring solutions- What if tools
- Composing artifacts and performances
- Reviewing and replaying session histories
- Disseminating results
Each activity in this broad agenda leads to a rich
literature and ample room for research. Projects at the
Human-Computer Interaction Laboratory are already
working on four of these activities: searching, consulting,
visualizing, and reviewing. This essay concentrates on
powerful software tools to support users in composing
artifacts and performances [4]. His framework and our
user needs assessments from several projects, especially
the Library of Congress and the World Bank, form the
basis for the analysis in this paper.
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
1
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
Genex Phase
Primary Activity (but can be used at any phase)
Collect
Searching and browsing digital libraries
Visualizing data and processes
Relate
Consulting with peers and mentors
Create
Thinking by free associations
Exploring solutions - What if tools
Composingartifacts and performances
Reviewing and replaying session histories
Donate
Disseminating results
Figure 1. Genex phases and their related primary activities
The activity of composing artifacts and performances is
further reduced into three stages: initiate, revise, and
evaluate (Figure 2). Many users initiate by collecting
exemplars, templates, and processes which are the basis
for their first steps. Then they need revision tools to
manipulate and refine, and finally they apply critiquing
and assessment tools to evaluate their compositions. Of
course, the process can be iterative, with users returning to
examine other exemplars and revising many times to reach
higher quality outcomes.
Composing Artifacts and Performances
Initiate
Review exemplars, templates and processes
Revise
Manipulate, refine, extend
Evaluate
Make consistent, repair flaws, assess outcome
Figure 2: Composing and its stages
The benefit of powerful tools is that novice users can
often produce results that rival the quality of what could
only be done by experts using more primitive tools.
However, there are multiple dangers. First, the tools
may constrain users into producing only a limited range
of possibilities. Rather than informing authors or artists
of a broader range of possibilities, the technology may
limit their imagination. In the worst case, users produce
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
2
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
something that is little more than painting-by-numbers.
Second, viewers of the polished products may be
deceived or misled in their assumptions of the skill,
knowledge, and capability of the authors. Authors who
build on previous work, but fail to credit their sources are
acting unethically. Similarly, viewers who are unaware
of the use of sophisticated tools may make naïve
judgments. Third, experts may threatened by the
proliferation of products that formerly required great
skill. Such transformations have happened throughout
history (e.g. artists who disparaged photography or
weavers who destroyed automated looms), but the
transition is often turbulent.
While keeping cognizant of these dangers, designers
can proceed to improve existing tools and create new
ones. Of course, a computer scientist’s approach might
be to consider recursion, and build a tool to build tools,
however we will leave that for a future date [20]. For
now, the focus of this paper is the narrow goal of
encouraging thoughtful design of software tools to
support the activity of composing artifacts and
performances.
2. Composition tools for initiation
Blank sheets of paper and a pencil are an invitation to
be creative.
Poets, playwrights, composers,
mathematicians, and artists have all appreciated its
freedom and open-ended possibilities. Paper and pencil
are responsive, since the users see results immediately,
and direct, since the users feel in control of a predictable
environment. However, paper and pencil limit artists'
revisions to what can be accomplished by additions and
erasures. Since artists want to explore alternative
lighting, texture, layouts, or sizes they often resort to
multiple sequential sketches. Repeated sketching is
productive since it refines their skill and compels them to
commit time to the task, during which their reflections
can lead to new insights.
A tool that enables artists to lighten an image, change
to a smoother texture, shift positions of components, or
enlarge individual features should support creative
exploration of alternatives [10]. Of course, modern
digital painting and drawing software enable just such
changes and many more, such as distortions that stretch
features in one direction, unusual effects such as
enhanced contrasts or negative reversals, and overlays of
multiple images. Actions can be applied to the whole
image or to selected components and combined actions
can produce amazing variations.
But great art is more than clever pixel manipulation.
Every work of art emerges from a rich context of culture
in which creative artists seek to engage viewers. Artists
want
to
startle/reassure,
enthrall/disturb,
or
remind/convince the viewers about classic themes such
as religion, war, love, or death or contemporary issues
such as civil rights, environmental preservation, or
consumerist society. Art can also be used for practical
purposes such as to sell products, promote events, or
decorate offices.
Understanding the interaction between the low-level
details and the high-level concepts is fundamental to
developing composition tools that support creative work
[8, 1, 2, 17, 18, 19]. Every artist who approaches a blank
sheet of paper brings a lifetime of learning about the
high-level concepts that are embedded in the artist's
culture and experience. The artist's intentions and goals
drive the process, but they are sometimes unnoticed by
naïve observers who see only paint or pixels.
Similarly for authors, word processing software easily
enables change to fonts, sizes, styles, or justification but
these low-level revisions are only part of the writing
process.
An effective writing environment would
support mid-level notions such as outlines, paragraph
structure, or consistent terminology and high-level
conceptualization of goals and intentions. Writing
teachers are clear that pre-writing tasks include
formation of the goals, statement of the audience, and
choice of rhetorical styles. These cannot be automated,
but they can be supported by providing authors with
exemplars, templates, and processes.
For example, the Microsoft PowerPoint slide
construction and presentation system begins by offering
users four choices (Figure 3):
- AutoContent Wizard: a process (Figure 4) that
guides users in composing slide shows on
topics such as a Financial Overview which
has these components: Agenda, Highlights,
Income, Revenue by Division, Revenue by
Location, Balance Sheet, Assets, etc.
- Template: a set of colorful backgrounds and
layouts for items in each slide. A powerful
feature is that users can work in one
template and then switch to another (Figure
5).
- Blank Presentation: a set of layouts for a single
slide that combine titles, lists of bulleted
items, charts, or pictures (Figure 6).
- Open an existing presentation: standard dialog
box for opening files, with the capacity to
search within a set of presentations (Figure
7).
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
3
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
Figure 3. Microsoft PowerPoint offers users four ways to begin work: An AutoContent Wizard
to design a full presentations, a Template to design a slide, a Blank presentation, and an
opportunity to open an existing presentation.
Figure 4: The Microsoft PowerPoint AutoContent Wizard provides starting sets of 5-15
slides that help organize a user’s presentation.
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
4
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
Figure 5: Microsoft PowerPoint’s second offering provides colorful templates (although the title should be
Template) for individual slides and allows users to switch templates even after they have created their
presentations.
Figure 6: Microsoft PowerPoint third offering provides Blank presentations (although
the title should be Blank Presentation) that guide users in formatting an individual slide.
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
5
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
Figure 7: Microsoft PowerPoint’s fourth offering enables users to locate and open an existing presentation.
Figure 8: Microsoft PowerPoint’s Menu Bar adheres to a standard, but provides
only marginal help to novices in converting their high-level goals into a sequence
of low-level actions.
Learn Collect Consult Initiate Revise Evaluate Publish Preserve
Figure 9: A new menu bar that was sequenced by the user’s experience might simplify
learning for novices.
These are powerful and helpful strategies, that can be
extended by template or layout authors within an
organization. PowerPoint is an impressive tool, but
missing features, a confusing menu structure, and the
obvious inconsistencies in layout, terminology, and form
might have been avoided if the designers had a more
comprehensive framework that guided their design. For
example, the menu bar (Figure 8) is more oriented
towards the designer’s conception of the tool, rather than
the user’s workflow. Many users struggle to find which
pull-down menu contains a desired feature. Of course,
Microsoft’s designers were constrained by existing
guidelines and commonality with other software
packages that all benefit from the standard File-EditView…Window-Help sequence. However, providing an
alternate menu bar or complementary navigation tools
has been done successfully by other designers.
One starting point for a redesign could be the fourphase genex framework of Collect-Relate-Create-Donate
or a more detailed sequence such as in Figure 9.
A meaningful and task-oriented sequence of menu bar
items is likely to help novice as well as experienced
users. With this detailed sequence users would probably
know that they should select Learn to begin by learning
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
6
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
about using the tools and getting help. Then the Collect
item would contain resources to search and browse to
find previous work, relevant web sites, books, maps, etc.
and tools for saving these resources or pointers. In
addition it might contain copyright tools to record use of
copyrighted material and contacts to request permission
to use the material. The next menu item, Consult,
reminds users that they can contact peers and mentors for
inspiration, feedback, and guidance. The next three items
clearly suggest the stages of creative work: Initiate
Revise Evaluate. Initiate would provide access to
exemplars, templates and processes and the blank
presentations. Revise would suggest low, middle and
high level ways to transform the presentation. Evaluate
would offer checking services and assessment of
outcomes. Finally Publish and Preserve remind users
that results need to be saved, disseminated to appropriate
readers in appropriate formats, backed up, and preserved
in archives. Interesting information such a log of to
whom copies were sent plus their feedback and
when/where the slide show was presented could also be
stored.
Window management features, which are actions on
the interface, should be separated from task actions, and
placed somewhere else on the window frame along with
scroll bars and rulers. Interfaces that enable users to
concentrate on their tasks and minimize distractions of
managing the interface are substantially more effective
and appreciated by users [16].
A menu bar with some sequence such as LearnCollect- Consult
-Initiate-Revise-Evaluate-PublishPreserve should be useful for most composition tools
such as word processors, spreadsheets, music, drawing,
or programming. Even more specialized versions for
specific products such as a resume, corporate annual
report, book review, or city guide could further help
focus the users’ actions. Instructive examples are
appearing in some software and web sites, such as
www.theglobe.com/homebuilder. This web site walks
users through the process of building their own web
pages for an event, a resume, or a fan club. They can
view samples and templates, and then be led through the
process in a step by step manner.
A much more ambitious tool, Dramatica Pro 3.0
(Screenplay Systems Inc, www.screenplay.com), helps
professional authors to compose screenplays [9]. It
comes with 67 templates from familiar movies, and
enables authors to initiate new screen plays by asking 50
question in the Quick Start approach or the more
advanced StoryGuide with 250 questions. Detailed
motivations of archetypical characters (protagonist,
antagonist, sidekick, guardian, etc.) and plot progressions
are coordinated with themes. It provides facilities for
revision and maintains some consistency checks, and
produces a variety of useful reports. In addition, it feeds
into a series of other packages to schedule, budget, and
produce movies.
The best creativity support tools enable users to make
low, middle and high level revisions (Table 1). Word
processors support low-level revisions to fonts, sizes,
styles, justification and other specific perceptual features.
They provide dialog boxes and tools to change middlelevel components such as outlines, style sheets, or other
features of a document. However, creativity support
tools are still weak in helping to revise the high-level
goals or plans of a document [11, 13]. Authors of a dance
composition tool stressed that “an important element of
the creative process is the need to move flexibly back
and forth between levels. Successive refinement of the
low level details may reveal the need to change the high
level theme [12].”
One approach to supporting high-level revisions is to
provide templates for specific kinds of documents such
as a resume. Then the users might be able to make
modest transformations such as going from reverse to
forward chronological order. Users could also shift the
emphasis from education to job experiences by changing
the order or adjusting the low-level features to give more
space and more prominent headings. Resume building
programs also assist users by providing reminders of
items to include such as awards, honors, prominent
references, or key accomplishments.
Interesting examples of highly specific support tools
are income tax preparation programs that guide users
with
numerous
questions
and
allow
legal
transformations. It may be extreme to call these
creativity support tools, but they are highly successful
products based on an intensely studied task. Therefore
they may provide insights for supporting creative tasks
such as writing a book review, scientific experiment
report, or a sonnet. Generators for each of these tasks are
available, but typically they guide authors through the
low to middle-level initiation and revision. Some critics
may complain that a template for a book review or a
rhyming dictionary cheapens the creative process, but the
outcome is still very much the author's product.
The examples show that there are already useful tools,
although their contribution is often limited to low and
middle levels. The framework may be useful in helping
designers see the full range of possibilities (Table 1).
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
7
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
3. Composition tools for revision
Table 1: A simple framework for characterizing multiple levels of revisions for words, images and music.
Words
Fonts, sizes, styles, margins,
indentation, justification, highlights,
white space
Images
Color, size, positions, texture,
shading, lighting, hue,
saturation, value
Music
Notes, amplitude, duration,
attack, instruments,
combinations
Middle
Outlines, paragraph structure, style
sheets, terminology, columns,
figures, tables
Layers, aspect ratio, grid,
cropping, perspective,
components, sharpness
Motif, movements, structure,
repetition, chords, harmonies
High
Goals, arguments, intention,
metaphors, rhetorical style,
authority, clarity
Purpose, graphical style,
message, impact
Genre, mood, tempo
Low
4. Composition tools for evaluation
As users compose and revise their artifacts or
performances, they may be helped by feedback. An
important method for feedback is consultation with peers
and mentors, which is another activity within the genex
framework. However, well-designed software tools can
provide some feedback as well. For example, modern
word processors contain spelling and grammar checkers
to offer suggestions of potential problems and repairs.
Most word processors also offer simple counts of
characters, words, lines, and paragraphs for those who
must write documents within certain length restrictions.
Some packages contain readability indexes, such as the
Flesch Index, tied to sentence length or word difficulty
ratings. These may also produce grade level indexes for
those who must adhere to such requirements. More
advanced tools might provide concordances with
feedback about the diversity of word usage or comments
on the excessive or sparse use of fonts, styles, etc. More
sophisticated analyses of high-level goals is possible in
constrained systems such as the tax preparation system
that continuously reports the amount of tax or refund as
revisions are made to income and deduction fields.
Similar strategies apply to preparing feedback for
screenplay writers, musicians, or artists. For example,
page layout programs may indicate production costs as
the number of pages grow, as binding options are
changed and as color page counts increase. Music
programs also keep track of the number of instruments or
tracks, and the duration of the performance. More
advanced evaluation features seem feasible, especially
when the application domain is constrained. Similarly
error, constraint and consistency checking are natural
next steps for many programs.
5. Conclusions
Software support for creativity is a challenging goal,
but designers can make progress by developing an
integrated framework such as genex and then focusing
on specific activities.
This essay dealt with the
composition activity and separated it into three stages:
initiate, revise, and evaluate. Existing system have some
useful features that support these stages, but concerted
efforts could lead to advanced prototypes with novel
features. These would have to be usability tested,
refined, and tested in field trials. Amplifying the abilities
of many people to be more creative more of the time
seems to be a worthy and attainable goal.
Acknowledgements
I appreciate the comments about creativity made by my
cousin Marius Sznajderman, a painter and printmaker,
during a leisurely May 1999 walk through Rock Creek
Park. Thanks to Jennifer Preece for suggestions on a
draft version.
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
8
Proceedings of the 33rd Hawaii International Conference on System Sciences - 2000
References
[1] Arnheim, R., Visual Thinking, University of
California Press, Berkeley, CA, (1969).
[2] Bertin, Jacques, Semiology of Graphics, University of
Wisconsin Press, Madison, WI (1983).
[3] Boden, Margaret, The Creative Mind: Myths &
Mechanisms, Basic Books, New York (1990).
[4] Brooks, Frederick, Jr., The computer scientist as toolsmith
II, Communications of the ACM 39, 3 (March 1996), 61-68.
[5] Couger, D., Creativity & Innovation in Information Systems
Organizations, Boyd & Fraser Publ. Co., Danvers, MA (1996).
[17] Tufte, Edward, The Visual Display of Quantitative
Information, Graphics Press, Cheshire, CT (1983).
[18] Tufte, Edward, Envisioning Information, Graphics Press,
Cheshire, CT (1990).
[19] Tufte, Edward, Visual Explanations: Images and
Quantities, Evidence and Narrative, Graphics Press, Cheshire,
CT (1997).
[20] Vlissides, John M. and Linton, Mark A., Unidraw: A
framework for building domain-specific graphical editors,
ACM Transactions on Information Systems 8, 3 (1990), 237268.
[6] Csikszentmihalyi, M., Creativity: Flow and the Psychology
of Discovery and Invention, HarperCollins, New York (1996).
[7] Gardner, Howard, Creating Minds: An Anatomy of
Creativity Seen Through the Lives of Freud, Einstein, Picasso,
Stravinsky, Eliot, Graham, and Gandhi, Basic Books, New
York (1994).
[8] Gombrich, E. H., Art and Illusion, Pantheon, New York
(1960).
[9] McKee, Robert, Story: Substance, Structure, Style and
Principles of Screenwriting, Harpercollins, New York (1997).
[10] Nakakoji, K., Suzuki, K., Ohkura, N., and Aoki, A., A
framework to support creativity in multimedia information
design, Proc. Human-Computer Interaction INTERACT'97,
Chapman & Hall, London (1997), 212-219.
[11] O'Malley, C and Sharples, M., Tools for management and
support of multiple constraints in a writer's assistant
Proceedings of the HCI'86 Conference on People and
Computers II British Computer Society (1986), 115-131.
[12] Schiphorst, T., Calvert, T., Lee, C., Welman, C., and
Gaudet, S., Tools for interaction with the creative process of
composition, Proceedings of ACM CHI'90 Conference on
Human Factors in Computing Systems (1990), 167-174
[13] Sharples, M. and van der Geest, T. (eds.) The New Writing
Environment: Writers at Work in a World of Technology
Springer-Verlag, London (1996).
[14] Shneiderman, Ben, Designing the User Interface:
Strategies for Effective Human-Computer Interaction: Third
Edition, Addison-Wesley Publ. Co., Reading, MA (1998).
[15] Shneiderman, B., Codex, memex, genex: The pursuit of
transformational technologies, International Journal of
Human-Computer Interaction 10, 2 (1998), 87-106.
[16] Shneiderman, B., Creating creativity: User interfaces for
supporting innovation, Univ. of Maryland Dept of Computer
Science Technical Report CS-TR-3988, UMIACS-TR-99-10,
ISR-TR-99-4 (1999).
0-7695-0493-0/00 $10.00 (c) 2000 IEEE
9