Visual Imagery and Consciousness
by
Nigel J.T. Thomas
California State University, Los Angeles,
Los Angeles, CA, USA
To appear in:
W.P. Banks (Ed.), Encyclopedia of Consciousness. Academic Press/Elsevier.
This penultimate draft ©Nigel J.T. Thomas, 2008.
Suggested keywords: image, visualization, consciousness, mental imagery, imagination, mind's
eye, perception, intentionality, imageless thought, quasi-pictorial, propositional, enactive,
sensorimotor, mental representation
Please note:
As is normal with commissioned pieces like this, there were some editorial restrictions on how it
should be written and structured. The title and topic were assigned to me (as well as the
requirement for a glossary), and if I had had my way I would not have restricted myself to just
visual imagery, but would have tried to explicitly cover imagery in other sense modes too. I believe
that all sensory modes of imagery are important in cognition and in our experience, and that our
consciousness of imagery in all modes should, and can, be understood within the same theoretical
framework. That said, however, visual imagery has been studied far more, and thus is better
understood, than imagery in any other mode, so it is not inappropriate to make it the main focus of
discussion.
But quite apart from this, the editorial policies of the Encyclopedia of Consciousness meant that
not only was I held to a quite strict length limit, and unable to use footnotes (or italics, for some
reason), but, more importantly, I was not permitted to give any citations within the text. In my
opinion, the latter was a very unfortunate and inappropriate restriction for an article dealing with
this sort of controversial subject matter (I protested to the editorial staff, but to no avail). The
"further reading" list that I was mandated to include in no way compensates for the lack of
appropriate in-text citations. The literature explicitly devoted to the problems of understanding
imagery as a conscious phenomenon is meager indeed. Most of the more important and influential
ideas about the subject are not to be found in works principally or even largely concerned with the
issue, but, rather, in brief passages, or sometimes just as implicit assumptions, within works whose
main focus lies elsewhere. In my view, the published article would have been much more useful to
readers if I had been able to provide copious citations.
I have been working on a revised version, to be made available both at the ASSC web site and my
own at http://www.imagery-imagination.com. I intend that it will include proper citations, as well as
notes that will expand upon, qualify, and, where appropriate, defend many of the claims that I
make. However, circumstances now make it unlikely that this can be made ready very soon, so I am
making this penultimate draft of the published version available here at the ASSC site now.
2
Visual Imagery and Consciousness
Nigel J.T. Thomas
California State University, Los Angeles,
Los Angeles, CA, USA
I.
II.
III.
IV.
V.
VI.
Introduction
Defining Imagery: Experience or Representation?
Historical Development of Ideas about Imagery
Subjective Individual Differences in Imagery Experience
Theories of Imagery, and their Implications for Consciousness
Conclusion
Glossary
Hard problem - In this article, the "hard problem" of consciousness refers to the problem of
understanding how a brain state (a spatio-temporal pattern of neuronal excitation, itself
reducible to a complex dance of molecules and ions around membranes) could possibly be a
conscious experience. Posed this way, the problem is unsolved, and may well be insoluble. It
does not necessarily follow that consciousness cannot be scientifically understood, however. The
problem may be ill-posed.
Imageless thought - A thought or thought content that is (i) consciously experienced, but (ii),
unlike mental imagery, has no sensory or "perception-like" character. The existence of such
thoughts has been highly controversial ever since the idea was first introduced into psychology
in the early twentieth century.
Intentionality - The property, possessed by many, perhaps all, mental acts or entities (such as
mental images, beliefs, desires, and thoughts in general), of being about, of, or directed at
something. The "thing" in question (sometimes referred to as the "intentional object" of the
image, thought, or whatever) may be real or imaginary (one may have a thought about a unicorn
quite as well as a thought about a horse). This technical, philosophical concept of intentionality
is only indirectly (if at all) related to the ordinary language notion of having an intention (to do
something), or of doing something intentionally (i.e, on purpose rather than inadvertently).
Mental Imagery - Quasi-perceptual experience: that is, experience that subjectively resembles
perceptual experience, but which occurs in the absence of the relevant perceptual stimuli. It is
generally acknowledged that imagery may occur in any sense mode – visual, auditory (including
what is sometimes called "inner speech," or "thinking in words"), olfactory, kinesthetic, etc. – or
even in several simultaneously. However, visual mental imagery, also colloquially referred to as
"visualization," "seeing in the mind's eye," "picturing," etc., has been by far the most extensively
atudied and discussed.
3
Mentalese - The hypothetical, innate and unconscious, "language of thought," held by some
cognitive scientists to be the symbolic format in which information is represented within the
brain, and which the brain, considered as a computer, uses in its computations. It supposedly
resembles the languages people actually speak in having a combinatorial syntax and an arbitrary
semantics. Clearly we are not consciously aware of our mentalese representations as such, but if
they do indeed exist they may be the vehicles of the conscious thoughts that we experience as
mental imagery and "inner" speech.
Retinotopically Mapped Visual Cortex - Certain visual processing areas of the cortex of the
brain, most notably the primary visual cortex (V1), that are structured as (rather distorted and
low resolution) maps of the light sensitive retina upon which light is focused at the back of the
eye. Adjacent regions of cortex, in these areas, correspond to adjacent areas on the retina such
that, during vision, the two-dimensional spatial pattern of excitation of the cortical neurons
corresponds topologically to the pattern of illumination, the optical image, on the retina.
Introduction
For most people visual mental imagery is a common, frequent experience. We often recall past
events, or imagine possible ones, by forming mental images of them. Our dreams also seem to
consist largely of mental imagery. Indeed, many philosophers and psychologists have held that,
together with immediate perceptual experience, imagery makes up the entirety of consciousness
(although clearly not just visual imagery, but also imagery in other sense modes, especially inner
speech, auditory or vocal-motor imagery spoken words). If we are to fully understand
consciousness, we will certainly need to understand mental imagery.
Philosophers have studied mental imagery for many centuries, experimental
psychologists have studied it for well over a hundred years, and more recently it has attracted the
attention of cognitive scientists and neuroscientists, but many basic questions still remain
unresolved. There is controversy not only over what cognitive and neural mechanisms are
responsible for imagery, but also over its function (if any) in cognition. Some regard mental
imagery (or some closely related notion, such as "perceptual symbols" or "image schemata") as
providing the necessary vehicle for all thought; some regard it as important only for certain types
of cognitive task (such as judging spatial relationships); and yet others regard it as a functionally
insignificant conscious epiphenomenon of the unconscious cognitive processes that really
constitute our thinking.
Like percepts, mental images bear intentionality. That is to say, they are always images
(or percepts) of something or other, of some (real or imaginary) "object." (Some philosophers
regard intentionality and consciousness as very closely related phenomena.) However, unlike
percepts, images occur in the absence of their object. You cannot perceive a cat when no cat is
present, but you can imagine a cat (or any other perceptible thing) at any time. Furthermore,
(mistakes aside) you cannot perceive a cat be other than where and how it actually is, but,
whenever you want to, you can imagine (i.e., form an image of) a cat as anywhere, or in any
4
condition. Images thus seem well suited to function as mental representations, allowing us to
think of things as they currently are not, and thus to recall the past, plan for the future, fantasize
about the unreal, and speculate about the unknown. Since ancient times, this has generally been
believed to be their cognitive function.
Defining Imagery: Experience or Representation?
Mental imagery is commonly defined as a form of experience: quasi-perceptual experience,
experience that subjectively resembles the experience we have when we actually perceive
something (see glossary). This implies that we are unlikely to be able to understand imaginal
consciousness unless we understand perceptual consciousness (and perhaps vice-versa).
Unfortunately, we do not yet have such an understanding (or, at least, one that is generally
agreed upon). It also, however, implies that imagery is always and necessarily conscious: if
something is not consciously experienced it cannot be mental imagery.
If, on the other hand, mental imagery is defined as being a form of mental representation,
as some contemporary cognitive theorists suggest, the tight conceptual linkage between imagery
and consciousness is broken, and it becomes conceivable that images (imaginal representations)
might sometimes play a role in cognition without our being consciously aware of them. There is
some evidence suggesting that this does indeed occur. For instance, experimental studies of
verbal memory have found that nouns for which it is easy to think of a corresponding image
(mostly words for concrete things, such as "dog," "ship," or "skyscraper") are more readily
remembered than nouns for which it is difficult to think of an image (mostly abstractions, such
as "truth," "nation," or "size"). However, this mnemonic effect of "imagability" seems to occur
quite regardless of whether any relevant images are actually consciously experienced (or, at
least, of whether the subjects report or recall experiencing them). One interpretation of this
finding is that image representations may be spontaneously evoked by the concrete words, and
may play a role in making these words more memorable, even when they do not rise to
consciousness.
Unlike the experiential conception of imagery with which we started, this
representational conception suggests that we might be able to understand the nature and function
of imagery without giving any attention to the fact that it is (at least sometimes) consciously
experienced, and, in fact, most empirical and theoretical cognitive science research on imagery
over the past few decades has proceeded on that assumption. The problem of the imaginal
consciousness has, very largely, been ignored or set aside, while questions and controversies
about its representational function (and, more recently, its neural instantiation) have been
pursued enthusiastically. Furthermore, little if any attention has been given to the question of
what might differentiate those cognitively active image representations that do enter
consciousness from those that do not.
But in any case, imagery probably cannot be satisfactorily characterized purely in terms
of its representational function. It is difficult or impossible to differentiate it from other forms of
5
actual or possible mental representation, without either, on the one hand, begging some very
controversial questions about its nature (about whether, for example, the relevant representations
are somehow picture-like), or, on the other hand, appealing to the experiential conception of
imagery. We could (and perhaps should) say that mental images are just those mental
representations whose presence to mind has the potential to give rise to quasi-perceptual
experiences. If so, however, the representational conception of imagery is made conceptually
dependent upon the experiential conception, and we cannot even begin truly to understand
imagery unless we take its conscious nature into account.
Historical Development of Ideas about Imagery
Contemporary scientific controversies about mental imagery by no means turn entirely on
matters of empirical fact. Disagreements turn largely upon conceptual issues, and upon differing
ideas about what questions a theory of imagery most needs to answer. For example, is it crucial
to consider the conscious and intentional nature of imagery when we seek to understand its
cognitive mechanisms and functions, or are such considerations irrelevant and distracting? An
awareness of the historical contexts from which the various contemporary research programs
emerged is indispensable in understanding and adjudicating between such differing perspectives.
Scholars disagree as to whether ancient thinkers such as Plato and Aristotle, whose works
contain the earliest known discussions of cognition, had anything like our modern concept of
consciousness. However, it is clear that they did have the concept of mental imagery. Plato
tentatively suggests that memory might be analogous to a block of wax into which our
perceptions and thoughts stamp impressions (i.e. memory images); he also speaks,
metaphorically, of an inner artist who paints pictures in the soul, and suggests that imagery may
be involved in the mechanisms by which the rational mind exerts its control over the animal
appetites.
However, it is Plato's successor Aristotle who provides the first systematic account of the
role of imagery in cognition. In Aristotelian psychological theory, images play much the same
role that the rather broader notion of mental representation plays in modern cognitive science.
He held that mental images play an essential role in memory and thought: memory is the recall
to mind of images of past events, and "It is impossible to think without an image". He also held
that images underpin the meaningfulness of language, and play a key role in motivation. Were it
not for mental images, he thinks, our speech would be empty noise, like coughing, and our desire
for (or fear of) something would only motivate our actions when that something was actually
being perceived. However, a mental image of the desired (or feared) thing enables us to think
about it in its absence, and thus can sustain our motivation at other times. Aristotle also posits a
mental faculty, phantasia (usually translated as "imagination"), that is closely allied to (or
perhaps even structurally identical with) the general faculty of sense perception, and is
responsible for creating our mental images. Some suggest that this faculty amounts to his
conception of consciousness.
6
In the wake of Aristotle's work, images generallycontinued to be seen as the principal
vehicles of mental content. This view lasted through later antiquity and the middle ages and into
the era of modern philosophy, as instigated by Descartes. In the work of Descartes and his
successors the mind was explicitly understood as conscious, and the contents of consciousness
were known as "ideas." Descartes himself seems to have at least two distinct concepts of "idea."
What he calls a "clear and distinct idea" is a direct (perhaps propositional) mental grasp of the
essence of something. It exists within consciousness itself, but is not an image . However, he
also hypothesizes that when we see, imagine, or remember something previously seen, a
pictorial image is formed deep within the brain (on the surface of the pineal gland). Such
images, which he also quite explicitly calls ideas, do not exist within the mind as such (since, for
Descartes, the conscious mind is immaterial, and distinct from the brain), but they are somehow
presented to the mind and are the immediate causes of our conscious perceptual, memory, and
imaginative experiences.
The British Empiricist philosophers' concept of "idea" seems to combine aspects of both
of the notions found in Descartes. Although some scholars today question whether John Locke
really believed ideas to be picture-like, it is natural to interpret him as thinking so, and there is
little room for doubt that his Empiricist successors, most notably Berkeley and Hume, conceived
of ideas as images. Hume distinguishes "impressions" (i.e., percepts) from "ideas" that arise from
the memory or the fancy, but he thinks that these differ only in the intensity with which we
experience them, and he seems to conceive of all visual impressions and ideas as consciously
experienced, picture-like images. Unlike the image-ideas of Descartes, however, for the
Empiricists image-ideas are themselves entirely mental, and exist only inasmuch as we are
conscious of them. Indeed, Hume's so called "bundle" theory of the mind suggests that these
images actually constitute consciousness: the mind is nothing but a bundle of impressions and
ideas.
Although it certainly had its critics, this Empiricist conception of mind continued to be
influential up to, and beyond, the emergence of scientific psychology in the late 19th century.
Pioneering experimental psychologists such as Wilhelm Wundt in Germany and William James
in the U.S.A. thought of psychology as the study of consciousness and of the images (and
emotional "feelings") that populate it. However, in the early 20th century a group of psychologists
based in Würzburg in Germany, led by Oswald Külpe, reported that systematic introspection of
thought processes under controlled laboratory conditions had led them to recognize that, as well
as images, the mind also contains "imageless thoughts." These were, supposedly, conscious
experiences (they had been discovered by introspection after all) and cognitive (they were
evoked during reasoning, and were not merely emotional feelings), but, unlike imagery, they had
no sensory character.
These claims proved extremely controversial. Some psychologists, such as Wundt,
argued that the introspective methodology employed by the Würzburg psychologists was
inherently unscientific and unreliable. Others, such as Edward Titchener, argued that
competently conducted introspective investigations revealed no evidence of imageless thoughts.
7
A bitter and irreconcilable dispute, the so called "imageless thought debate," developed within
psychology. It is important to be clear that no party to this debate was claiming that thought in
general is imageless; rather, the point at issue was whether any non-imaginal, but nevertheless
conscious, thought contents exist. Although the factious arguments about this have long since
died down, the issue has never truly been resolved. Some people still hold (although perhaps
more often as an implicit assumption, rather than an explicitly defended view) that all the
cognitive contents of consciousness have a sensory character, either as actual sensations or
percepts, or else as mental images; others hold (again, often implicitly) that there are also nonsensory conscious thought contents. These latter are sometimes referred to as states of "fringe
consciousness," and may be described as "feelings" of, for example, "familiarity," "rightness,"
"confidence," etc., (or their opposites).
Historically, however, the upshot of the deadlock of the imageless thought debate was a
general discrediting of introspective methodologies in psychology, and even of the very idea that
consciousness could be studied scientifically. John B. Watson argued that consciousness was an
inherently unscientific notion, and, as a central plank of this argument, questioned the very
existence of mental imagery. He urged that psychology should be reconceived as the study of
behavior rather than the study of consciousness, and the Behaviorist movement that he instigated
came to dominate the field for the next several decades (very roughly, from about 1920 to about
1960). Few, if any, of the Behaviorist psychologists who succeeded Watson went quite as far as
he did toward denying the reality of consciousness and imagery, but almost all of them, in
practice, treated them as beyond the reach of science. During the same period, the rise of the
analytical philosophy movement (with its emphasis on formal logic and language as the keys to
understanding knowledge and the mind), and particularly the influence of Ludwig Wittgenstein,
also led many philosophers to discount the significance of imagery, or simply to ignore it. The
upshot was that neither imagery nor consciousness received much serious attention during the
Behaviorist era.
A revival of interest in imagery, however, was an important component of the cognitive
psychology movement that challenged and eventually displaced Behaviorism as the dominant
psychological paradigm in the 1960s and '70s. New experimental methods for studying imagery
were devised that did not depend on purely subjective introspection, and evidence emerged
suggesting that imagery plays a large role both in memory (as shown mainly be experiments on
verbal memory), and spatial thinking (as shown by experiments on "mental rotation" and
"mental scanning"). A strong empirical case was built up for the functional importance of
imagery in cognition, and a vigorous (and continuing) debate ensued about the nature of the
cognitive mechanisms responsible for imagery.
However, there was not a comparable revival of scientific interest in consciousness until
the 1990s. As cognitive theories of imagery developed in the 1960s through the 1980s and
beyond, only cursory attention was given to the fact that it is a conscious phenomenon (or to the,
arguably, closely related fact that it bears intentionality). Even today (the early 21st century),
cognitive scientists and neuroscientists often treat imagery purely as a form of representation,
8
and offer no substantive account of how images are able to represent (almost certainly it is not
because they resemble their objects), or of how they come to be consciously experienced.
Subjective Individual Differences in Imagery Experience
Since the pioneering work of Francis Galton in the nineteenth century there has been a tradition
of research into individual differences in the subjective experience of imagery. Galton prepared
a questionnaire in which he asked his subjects to recall their morning breakfast table, and to
consider "the picture that rises before your mind's eye." They were then asked to comment on a
number of subjective aspects of that image, such as its brightness, its clarity, and the distinctness
of the colors. One of the best knowing findings of Galton's study is that some small minority of
the questionnaire respondents reported that they were unable to visualize anything whatsoever.
Unfortunately, no systematic research has since been done on this phenomenon, and the issue
remains very poorly understood (although recent research has directly contradicted Galton's
related claim that scientists are particularly likely to be weak or "non-" imagers).
Subsequent researchers have refined Galton's questionnaire technique in various ways,
most importantly, perhaps, by introducing numerical scales along which the subjects can rate the
strength of the image attribute of interest. Most often, this attribute is "vividness," but there have
also been attempts to use the questionnaire technique to measure such things as how easily
people can transform their mental images (by asking them to form an image and then change it
in some specified way, and then rate the difficulty of doing so), or how frequently they use
images of one or another sense mode in their thinking. In the early 20th century there was
considerable interest in classifying people into "imagery types" according to which sensory mode
of imagery (visual, auditory, haptic, etc.) they preferred. It was hoped that these "types" might
prove to be correlated with other independently measurable aspects of the person's psychological
makeup. However, no clear cut pattern of findings emerged.
Perhaps the most extensively and successfully used imagery questionnaire of recent times
is the Vividness of Visual Imagery Questionnaire (VVIQ) devised by David Marks. Taking the
VVIQ involves being asked to visualize a series of specified objects or scenes and to rate the
vividness of each resulting image on a five point scale ranging from "no image at all" to "as clear
and vivid as normal vision." An average score is then calculated.
Clearly research of this sort deals with imagery as a conscious phenomenon. It relies
upon the subjects' verbal, introspective reports of their subjective experience. Furthermore, there
has been much effort to find correlations between a people's scores on imagery questionnaires
and other psychological variables. The results, however, have generally been disappointing.
Even where reproducible correlations have been found, it has often proven difficult to make
much theoretical sense out of them. For example, there is some evidence that vivid imagers (as
measured by the VVIQ) remember pictures (color photographs) rather better than less vivid
imagers; however, surprisingly, it has also been found that the less vivid imagers (by the VVIQ
again) actually remember specific shades and hues of color better. Some research suggests that
9
when people form mental images they have elevated levels of activity in the retinotopically
mapped visual cortex of their brain, and that this activity is greater for more vivid imagers.
However, other researchers fail to find any elevated levels of activity in these areas during
imagery (they find elsewhere in the brain). Perhaps most surprisingly and disappointingly,
however, virtually no sign of any correlation has been found between people's vividness ratings
and their performance (speed or accuracy) in various visuo-spatial thinking and problem solving
tasks, even though, subjectively, such tasks very much depend on imagery.
There are a number of conceptual concerns about the validity of questionnaires of this
type. Most obviously, as the rating is necessarily purely subjective, and as nobody can
experience another's image, there is no way to tell whether the scales are being applied in a
consistent way. Perhaps an image that one person thinks of as "clear and reasonably vivid"
another might rate as "vague and dim," simply because different subjective standards are being
applied.
Furthermore, most people will probably agree that the vividness of their imagery can vary
markedly from time to time, circumstance to circumstance, and image to image: Some memories
may come back to us (often for no very apparent reason) as especially vivid images, whereas
others (or even the same ones on other occasions) are recalled only vaguely and dimly. Thus it
may be that these tests do not measure a person's true capacity for having vivid images, but only,
at best, the vividness that their images tend to have under the circumstances of filling out an
imagery questionnaire.
Finally, it is unclear quite what "vividness" truly means; whether it is really a well
defined, coherent attribute of experience. Perhaps different subjective image features such as
(for example) clarity, apparent brightness, level of discriminable detail, stability of the image
(whether it can be held in consciousness for a time, or quickly fades), and so forth, in fact vary
independently of one another, but tend to get indiscriminately lumped together as "vividness."
Some recent research suggests that introspective ratings of certain more fine grained aspects of
subjective imagery experience may be more meaningful than simple vividness ratings.
Theories of Imagery, and their Implications for Consciousness
There is much controversy in the contemporary scientific and philosophical literature about the
underlying nature and mechanisms of imagery. The rival theories fall into three broad classes,
each of which have distinctive implications for our understanding of imaginal consciousness:
! Picture theory (its contemporary advocates often prefer to call it "quasi-pictorial," or
"analog" theory)
! Description theory (also, rather misleadingly, known as "propositional" theory)
! Enactive theory (there is no established name for this type of theory; different versions
are known as "motor theory," "sensorimotor theory," "perceptual cycle theory,"role taking
theory," and "perceptual activity theory".)
10
Because of the vigorous, high-profile dispute (known as the "analog-propositional"
debate, or sometimes just "the imagery debate") that flared up between advocates of the first two
theory types in the 1970s, and that continues to rumble on even today, picture and description
theories have become by far the better known of these theoretical alternatives. Unfortunately,
this debate been conducted almost entirely without regard to the conscious nature (or, indeed,
the intentionality) of imagery. Each of the three theory types purport to be able to account for the
full range of established empirical facts about imagery, and, in fact, each is flexibly enough
conceived that it is likely to prove difficult to choose between them on narrowly empirical
grounds. That being so, it seems reasonable that our evaluation of them should look to broader
theoretical concerns, such as their respective implications for the theory of consciousness.
Picture theory
Picture theory, as the name suggests, holds that visual mental images may be identified with
representations in the mind and/or brain that are in some significant way like pictures, or, at
least, that represent things in much the same way that pictures represent things. (Whatever that
way may be. Although it is widely believed that pictures represent through resemblance, it does
not seem likely that resemblance can be the basis of mental representaton. Resemblance is a
fundamentaly subjective relationship that is recognized when a conscious mind focuses upon
certain similarities between things while, at the same time, discounting or ignoring other,
equally objectively real dissimilarities. In order to do this, the mind must already be able to
represent the things, and their relevant aspects, to itself.)
Picture theory is undoubtedly the oldest and most widely accepted theory of imagery.
After all, since prehistoric times people have used pictures as a means of causing one another to
have visual experiences of things that are not really there, and the analogy with mental imagery
(also experience of things that are not really there) is all too easily drawn. The theory goes back
at least to Plato, and some rudimentary version of it seems to be deeply entrenched in "folk"
(i.e., lay or commonsense) beliefs about how the mind works. Indeed, many of the expressions
used to talk about imagery in ordinary colloquial language (such as, "mental picture," "the mind's
eye," and even "image" itself) seem to be derived from this "folk" version of picture theory. The
fact that the theory is embedded in our language in this way can make it difficult to express or
understand criticisms of it, or to clearly grasp alternative accounts of the actual phenomena.
Nevertheless, many twentieth and twenty-first century philosophers have raised telling
objections to it, leading some of them to be accused (quite unjustifiably) of denying or doubting
that people have quasi-visual experiences.
However, the picture theory today is not just a folk theory. From the 1970s onward,
cognitive scientists, most particularly Stephen Kosslyn, have developed the basic idea of the
mental image as an inner picture into a sophisticated and detailed scientific model designed to
account for a wide range of experimental findings. More recent versions of the theory hold that
the visual mental image is embodied as a spatially extended two-dimensional pattern of neural
excitation in the retinotopically mapped visual cortices of the brain. Such patterns, isomorphic to
11
the pattern of optical stimulation on the retinae of the eyes, are known to occur during vision.
Kosslyn holds that our mental images are similar patterns, but generated from internal sources
rather than sensory input.
This is not the place to review all the alleged empirical shortcomings of quasi-pictorial
theory. However, both introspective and empirical evidence suggest a number of ways in which
the experience of imagery and the experience of looking at a picture vary. Perhaps the strongest
such evidence comes from experiments involving pictures that invite more than one
interpretation, such as the Necker cube and the duck-rabbit (figure 1). Subjects find it very much
harder to see both interpretations in their mental images of these than they do when the picture is
physically displayed in front them, and this applies even when the picture is their own drawing,
based upon their mental image.
<Figure 1 near here. Caption: The Necker cube and the duck-rabbit.>
However, Kosslyn maintains that the pattern of neural excitation that (he thinks)
constitutes the mental image is only a picture in an extended, metaphorical sense: a "quasipicture." After all, unlike regular pictures (drawings, photographs, projected optical images,
etc.), we do not need to see it with our eyes in order to derive information from it, or consciously
experience it. The neural excitation pattern is like a picture only inasmuch as it represents spatial
relationships within the two-dimensional projection of the represented visual scene by
topologically equivalent spatial relationships in the spatially extended pattern of excitation.
Kosslyn maintains that because his theory does not identify mental images with pictures in the
full, literal sense, but only with these "quasi-pictures," it avoids all of the many powerful
objections that have been raised against naïve "folk" versions of picture theory. Whether this is
so, however, remains controversial.
Although the modern version of pictorial theory can account for a large amount of
empirical data about how people use mental images as representations, its supporters have given
very little attention to the question of how it might account for the fact that imagery is (at least
sometimes) consciously experienced. There seem to be two basic ways in which this issue might
be approached. Either the quasi-picture in the cortex might be conscious in and of itself, or,
alternatively, consciousness arises when some other, more inward cognitive structure, a "mind's
eye" as it were, somehow "quasi-sees" the quasi-picture.
The first option, however, runs immediately into the notorious "hard problem" of
consciousness in its most intractable form: we really have no idea how a pattern of excitation in
part of the brain (excitation that is ultimately nothing but an elaborately choreographed dance of
molecules and ions around and through membranes) could possibly be, or give rise to, a
conscious experience of any sort, let alone an experience of a spatially extended structure
isomorphic to the excited brain region.
12
In any case, the second option appears to be favored by Kosslyn, the leading
contemporary advocate of picture theory. He speaks of (and diagrams) a "mind's eye function"
examining the quasi-pictorial image in the brain, and some of his explanations of empirical
findings seem to depend upon this idea. Perhaps then, just as ordinary visual consciousness
arises when people take in visual information with their eyes, imaginal consciousness arises
when the mind's eye takes in visual information from the inner picture.
But the "mind's eye" strongly suggests that the theory commits the homunculus fallacy.
To whom might this eye belong if not to a conscious little man inside my head, who experiences
the inner world of quasi-pictures in a way analogous to that in which I, a whole person,
experience the external world through my bodily eyes? Compounding this worry is the fact that,
at a formal, structural level (abstracting away from implementational details), Kosslyn's quasipictorial theory of imagery looks very like a more detailed version of Descartes' theory of
imagery. For Descartes too, mental images are material quasi-pictures formed within the brain
and consciously experienced not simply because they are there, but because they are somehow
"seen" by the soul. Kosslyn makes it clear that he does not intend to follow Descartes in
attributing our consciousness of these brain pictures to an immaterial soul (a ghostly
homunculus) beyond the reach of science; but although quasi-pictorial theory may not
necessarily imply Cartesian dualism, it does seem to be at least committed to Cartesian
materialism, the notion that subjective experience depends upon some structure in the brain that
(like a soul or a homunculus) acts as a conscious inner spectator of internal representations.
Quasi-pictorialists seem to have two comebacks to the homunculus objection. One is to
point out (as Kosslyn has often done) that a computer simulation of quasi-pictorial theory has
been implemented, and that there can be no homunculus (at least of any objectionable sort)
within a computer. Unfortunately, however, it turns out that the computer program in question
makes no attempt whatsoever to model the conscious nature of imagery. Like most of Kosslyn's
work, it is concerned to explore how certain sorts of spatial thinking might be achieved through
imagery, via such operations as scanning linearly across a quasi-picture, or rotating it into a new
orientation. The program performs such operations on internal data structures that are supposed
to model, at a functional level, the brain quasi-pictures posited by the theory, and it displays its
results in the form of actual pictures that are rotated or shifted across the computer's display
screen in relevant ways. No attempt is made to simulate the hypothesized "mind's eye function,"
and consciousness, either real or simulated, does not enter into the matter at all unless and until
some human operator looks at the screen. It is true that there is no homunculus in the computer,
but, inasmuch as the system models consciousness at all, it models it by co-opting a full sized
conscious human being to play the the homunculus role.
Not coincidentally, the program also omits to model the intentionality of imagery. The
fact that the two-dimensional patterns it produces and manipulates may look like pictorial
representations to human onlookers (and were designed to look that way by the programmers) is
quite irrelevant to the program's functioning. For the computer, they represent nothing.
13
The second comeback, however, is not dismissed quite so easily. Perhaps cognitive
science can render the homunculus and its "mind's eye" innocuous by showing that it can be
reduced to a set of computational or neural processes. Quasi-pictorial theory was originally
developed in the context of the "information processing" paradigm which dominated perceptual
theory at the time (the 1970s), and which still deeply influences the way many cognitive
scientists think about perception. In essence, "information processing" theory regards vision as a
one way flow of visual information, in through the eyes and then through a series of processing
stages in the brain until it is eventually transformed into a representation, or a set of
representations, suitable for guiding behavior. (This is an oversimplification, but not, in this
context, a misleading one. Developed information processing theories generally call for a
significant degree of top-down modulation of the bottom-up flow of information from the sense
organs. Nevertheless, the bottom-up flow is seen as dominating and driving perception.) A
common assumption (usually implicit) is that this final set of representations, this ultimate
product of the visual information processing system, is the immediate cause of conscious visual
experience.
(Some information processing theorists may prefer to think of visual consciousness not
as something attaching to representations, but, rather, as something arising from the workings of
the processing system as a whole, or even of the whole organism, including the sense organs and
the muscles that support behavioral response. However, this approach is not open to the pictorial
imagery theorist, who is directly committed to holding that we have conscious experience of
inner representations.)
According to quasi-pictorial imagery theory, one of the earlier stages of this visual
information processing is the creation of quasi-pictorial representations in the brain. These may
be derived from actual present visual input from the eyes (when we are actually seeing), or they
may be created from stored data in memory (when we are remembering or imagining), but in
either case they must be passed through several more stages of processing in order to extract
useful information from them. It is these further stages that constitute the "mind's eye function."
Thus, it can be argued, the mind's eye has a principled and independently motivated role within
the broader theory of vision. As scientists are actively investigating and seem to be making
progress in understanding the computational and neural mechanisms of visual information
processing, this "mind's eye" function is not so much a non-explanatory homunculus as a
promissory note drawn against the expected success of an ongoing research program.
But even if information processing theory does provide the right framework for
understanding visual perception (and not everyone thinks it does), the output of all the
processing is just more representations, instantiated as patterns of neural activity. Once again, it
seems that we must say either that these representations are conscious in and of themselves,
which brings us smack up against the "hard problem" once again, or else we need another
homunculus to read them and be conscious of what they represent. Perhaps this homunculus,
also, might be reduced to a further series of stages of information processing, but this would only
14
lead us to the same place yet again, and so on in unending regress. We still do not begin to
understand how quasi-pictures could be, or could produce, conscious imaginal experiences.
Description theory
Description theory, whose most important advocate has been Zenon Pylyshyn, began as an
attempt to understand how the phenomena of imagery could be fitted into a computational
theory of the mind. Pylyshyn holds that the way computers (and, by extension, brains) represent
information is necessarily more like language than like pictures. The syntax, and, indeed, the
vocabulary, of the hypothetical internal brain language (sometimes called "mentalese") might be
very different from that of any language that anyone speaks, but it is language-like because it
ultimately consists of symbolic tokens that represent things in the world in much the same way
as the words of a language do, not through resemblance but through some sort of arbitrary
correspondence relation.
In the case of "natural" languages, like English or Chinese, this correspondence (which
determines, for instance, that the word "dog" refers to dogs) is established by social convention.
In the case of humanly written computer programs the external reference of symbolic tokens
(when they have one) is set by the programmer. Matters are much less clear when we come to
hypothetical symbolic tokens in the brain, and there is great controversy over how, or whether,
the "symbol grounding problem" (the problem of understanding how the "words" of mentalese
might be able to refer to things outside the brain) can be solved. Nevertheless, much work in
cognitive science proceeds on the assumption that a solution is there to be found.
If the symbolic tokens that comprise computational representational systems represent as
words do, then it seems to follow that, by analogy with the way language represents visual
scenes, the brain-computer represents visual scenes with descriptions. From this perspective,
such mentalese descriptions are the end-product of perceptual information processing, and thus
responsible for our perceptual experiences. Of course, we do not experience the descriptions as
being descriptions, but, it is assumed, they nevertheless constitute our perceptual experience. If
mentalese descriptions of visual scenes are retrieved from memory rather than being the result of
present sensory input, or if they are constructed out of bits and pieces of various descriptions in
memory, then we have the experience of mental imagery.
Pylyshyn claims that, since descriptions can be partial, incomplete, and can leave out all
sorts of information (not only details, but sometimes even facts about the global structure of a
scene) description theory can explain the frequently indefinite and ambiguous nature of our
imagery better than picture theory can. However, it is clear that the theory's main motivation is
the belief that it better respects fundamental facts about the nature of computational
representation.
If anything, however, description theorists have paid even less heed than quasi-pictorial
theorists to the question of how images (or, come to that, percepts) come to be consciously
15
experienced. Clearly, we are not aware of mentalese descriptions as such, but the idea seems to
be that we can, at least sometimes, be consciously aware of what they represent. Like the quasipictorialists, however, description theorists appear to be implicitly committed to the view that
mental representations, including the elementary symbolic tokens of mentalese, and the symbol
complexes that make up visual descriptions (and, thus, mental images), are physically
instantiated as patterns of brain excitation (just as representations in a computer are instantiated
as electrical charge patterns in RAM chips and CPU registers). This brings us to exactly the
same "hard problem" as before: it seems impossible to conceive how patterns of brain excitation
(which, themselves, are reducible to movements of ions, etc.) could be, or give rise to, conscious
experience as we know it.
Enactive Theory
Quasi-pictorial and description theories are both attempts to explain how imagery, quasi-visual
experience, might be explained within the context of an information processing theory of
perception. They disagree over whether it should be identified with representations frm an early
or late stage of visual processing, and over the format of those representations. The enactive
theory of imagery depends upon a quite different way of conceiving of perception, one that was
pioneered in the twentieth century by James Gibson, and is more recently exemplified in "active
vision" techniques used in robotics, and the "sensorimotor" or "enactive" theory of perception
advocated by J. K. O'Regan and Alva Noë, amongst others. Instead of regarding vision as, at
root, a matter of information flowing in through the eyes into the brain, and toward some
internal center of consciousness, enactive theory regards it as a matter of the visual system
actively seeking out and extracting (or "picking up") desired information from the environment.
Seeing is not like taking a photograph (even a digital photograph whose file gets passed on to a
computer for further processing); it is more akin to performing a series of (many, rapid)
scientific tests and measurements on the information bearing ambient light that surrounds us
(what Gibson called the "optic array"). Rather than being the passive reception of information,
vision (and perception in general) is a purposive process of, as it were, asking questions about
our surroundings, and actively seeking out the answers. Visual consciousness is the experience
of being engaged in this exploratory, questioning, information seeking activity, involving not
only the making of the tests and measurements themselves, but also the continual adjustment of
our expectations, and thus our exploratory behavior itself, in response to their findings.
Consider how we identify an object by touch. If something is simply pressed against our
skin, we can tell very little about it. However, if we actively explore it, moving our fingers
around to feel its shape and texture, seeking out corners and edges, squeezing it to assess its
hardness, and so on, we can discover a great deal. What we learn derives not simply from the
sensations we feel in our skin, but, crucially, from the way those sensations change in response
to the purposeful, information-seeking movement of our fingers. Enactive visual theory holds
that vision works in fundamentally the same sort of way, except that the exploratory,
information-seeking behavior mostly happens so quickly and automatically that its details are
unavailable to introspection.
16
Eye movements are perhaps the most obvious (and experimentally accessible) aspect of
this visual information-seeking behavior – our eyes constantly, and purposefully, but largely
unconsciously, flit rapidly around to take in information from different points of interest – and
the enactive theory of imagery finds support in a number of recent experiments showing that the
stimulus-specific eye movement patterns produced when a subject examines a complex visual
stimulus are (quite unconsciously) re-enacted when the same subject later forms a memory
image of that stimulus.
However, eye movements cannot be the whole story. There clearly are bottom-up, inner
representations involved in human vision. For example, the pattern of illumination on the retina
does indeed produce a corresponding pattern of excitation in the primary visual cortex, and this
carries information about the scene momentarily before our eyes. From the enactivist
perspective, however, such representations are not conscious or even mental in any very
meaningful sense. Like the optical image formed inside a camera, the excitation pattern of
primary visual cortex carries data about our surroundings, but this only gives rise to visual
knowledge and awareness inasmuch as the data is purposefully explored and queried. It is this
active, exploratory questioning of a data source that turns the mere passive reception of visual
information (something that a camera can do) into true perception.
It is important to note, also, that the particular pattern of this exploratory questioning,
and the particular tests and measurements brought into play (both external eye movements and
internal data analysis processes), will be quite different when different sorts of things are being
looked at. For example, in order to see a cat we would need to go through a set of exploratory
actions appropriate to cat-seeing. These might include moving our eyes to focus on the likely
location of particular characteristic cat features (pointed ears, gently curving tail, etc.), but will
also undoubtedly include purely internal processes whereby further questions are answered by
querying the data flowing bottom-up into the brain. The key idea is that there is a specific
structured set of exploratory queries, what has been called a visual routine, appropriate to cat
seeing (or even Kitty-curled-up-asleep seeing), and a different set for each other type of thing
that we are able to recognise.
So, if visual experience arises from the active seeking for, and finding, of information,
imagery, quasi-visual experience, arises when we actively seek certain information, and persist
in going through the motions of looking for it, even though it is not there to be found. To
visually recognize a cat is to run through a specific, cat-recognizing visual routine, and to
imagine a cat is to perform (or partially perform) this same visual routine when there is no cat
present. The experience is different from actual seeing because there is no perceptual feedback
from the cat itself (and we must, as it were, force ourselves to go on looking for cat features
even though it is becoming ever more clear that we will not find them) but it is similar to real
cat-seeing inasmuch as the pattern of perceptual exploration that we perform is similar.
Arguably, this reflects the phenomenology of imagery, which is subjectively both like and unlike
true seeing.
17
From this theoretical perspective, to have a conscious visual experience is not to have a
representation in one's brain, but to act, and to be interacting with the world, in a certain way. To
have imagery is to act in the same sort of way, but to fail to interact. Actions (unlike brain states)
have inherent intentionality, just as mental images do: actions are necessarily about, or directed
at, something (possibly something imaginary, as when we search for the leprechaun's gold), just
as images are always images of something. When I enact my perceptual routine for looking at,
and recognizing, a cat, my action is intended to enable me to see a cat, and in a real sense it
remains so, it remains cat-directed, even when I enact it knowing that no cat is actually there to
be seen.
However, to point out that actions have intentionality is not, thereby, to explain how they
come to have that intentionality, and to say that mental images are really actions rather than
entities (representational brain states) is not, in itself, to explain how they come to be conscious.
Much more work needs to be done to make these matters clear. Although it has been argued that
enactive (or "sensorimotor") perceptual theory can explain perceptual consciousness, such
claims remain very controversial. Nevertheless, the enactive theory of imagery clearly radically
reframes the problem of explaining imaginal consciousness. No longer is it (as for quasipictorial and description theories) a problem of explaining how a brain state can be a conscious
(and intentional) state. Rather the problem is to give a scientific account of action and its
intentionality (a problem we faced anyway), and then to explain how such action can somehow
constitute consciousness. This is, no doubt, a difficult problem, but it is not the same as the
"hard problem."
Conclusion
At the time of writing, imagery is still a rather neglected topic within the broader, and growing,
field of consciousness studies. This is in rather stark contrast to the importance it was accorded
by earlier, pre-Behaviorist students of consciousness. Indeed, the Behaviorist revolt against
consciousness, which let to several decades of scientific neglect of the topic, seems to have been
fueled, in considerable part, by frustration at the factious and irreconcilable "imageless thought"
controversy, and it clearly involved the rejection of imagery quite as much as the rejection of
consciousness itself. Perhaps recent students of consciousness have been reluctant to fully
engage the topic of imagery because it has already proven so controversial within cognitive
science, and because the best known theories that have been developed in that context do little to
illuminate imagery's conscious nature. However, this situation may now be changing, and a
developing understanding of imagery may come, once again, to be seen as an integral and
essential aspect of our developing understanding of consciousness itself.
Further Reading
Baars, B. J. (ed.) (1996). Mental imagery. Special issue of the journal Consciousness and
Cognition, 5 (#3).
18
Berman, D. and Lyons, W. (2007). The first modern battle for consciousness: J. B. Watson's
rejection of mental images. Journal of Consciousness Studies, 14 (#11), 5-26.
Dean, G. M. & Morris, P. E. (2003). The relationship between self-reports of imagery and spatial
ability. British Journal of Psychology, 94, 245–273.
Ellis, R. D. (1995). Questioning consciousness: the interplay of imagery, cognition, and emotion
in the human brain. Amsterdam: John Benjamins.
Kosslyn, S. M., Thompson, W. L. and Ganis, G. (2006). The case for mental imagery. Oxford:
Oxford University Press.
Marks, D. F. (1999). Consciousness, mental imagery and action. British Journal of Psychology,
90, 567-585.
Morris, P. E. and Hampson, P. J. (1983). Imagery and consciousness. London: Academic Press.
O'Regan, J. K. and Noë, A. (2001). A sensorimotor account of vision and visual consciousness.
Behavioral and Brain Sciences, 24, 939-1031.
Pylyshyn, Z. W. (2002). Mental imagery: in search of a theory. Behavioral and Brain Sciences,
25, 157-237.
Richardson, J. T. E. (1999). Mental imagery. Hove, U.K.: Psychology Press.
Sartre, J.-P. (1940). The psychology of imagination. (Translated from the French by B.
Frechtman, New York: Philosophical Library, 1948.)
Schwitzgebel, E. (2002). How well do we know our own conscious experience? The case of
visual imagery. Journal of Consciousness Studies, 9 (#v-vi), 35-53.
Thomas, N. J. T. (1999). Are theories of imagery theories of imagination? An active perception
approach to conscious mental content. Cognitive Science, 23, 207-245.
Thomas, N. J. T. (2007). Mental imagery. In Zalta, E. N. (ed.) The Stanford encyclopedia of
philosophy. Stanford, CA: CSLI. Online publication:
http://plato.stanford.edu/entries/mental-imagery/
Thompson, E. (2007). Look again: phenomenology and mental imagery. Phenomenology and the
Cognitive Sciences, 6, 137-170.
Tye, M. (1991). The imagery debate. Cambridge, MA: MIT Press.