Mental Representation of Navigation through Sound-Based Virtual
Environments
Jaime Sánchez
Department of Computer Science, University of Chile, Santiago, Chile
Gloria Noriega
Department of Computer Science, University of Chile, Santiago, Chile
Carolina Farías
Department of Computer Science, University of Chile, Santiago, Chile
Please address all correspondence to:
Jaime Sánchez
University of Chile
Department of Computer Science
Blanco Encalada 2120, Zip Code 2777, Santiago, Chile
Phone Number: 56-2-9780502
Fax Number: 56-2-6731297
E-mail address: [email protected]
Paper presented at the
Annual Meeting of the American Educational Research Association
New York City, March 24-28, 2008
1
Mental Representation of Navigation through Sound-Based Virtual
Environments
Jaime Sánchez
Department of Computer Science, University of Chile, Santiago, Chile
Gloria Noriega
Department of Computer Science, University of Chile, Santiago, Chile
Carolina Farías
Department of Computer Science, University of Chile, Santiago, Chile
Various studies have analyzed the capacities that blind people possess for spatial representation,
utilizing movement, verbal descriptions and tactile maps. We have investigated the mental image
that blind users manage to model the navigated space using virtual environments supported by
specialized sounds, obtaining revealing results. This paper consists of a comparative study
between the representation of restricted navigation virtual spaces and free navigation virtual
spaces. To these ends, users have interacted with AudioDoomII and AudioLink, both soundbased virtual worlds. For each of the environments the participants were asked to form multiple
concrete representations and then evaluated and compared. The results demonstrate that a
virtual world with more highly restricted navigation is represented with a higher degree of
exactitude than a world with a freer navigation. The latter is more similar to the real world.
Keywords: Blind users, spatial sound, cognitive maps, spatial representation, virtual
navigation, open and restricted virtual environments.
Introduction
Spatial knowledge and cognition are important for navigation and mobilization in blind
people. Carreiras & Codina (1994) confirm that blind people are capable of construing spatial
images from verbal descriptions, although their level of success is less and requires more time
than a sighted person. Another study (Ungar, Blades & Spencer, 1996) concludes that, as it may
be true that visual experience facilitates the construction of spatial representations, it does not
guarantee that the totality of the information from a given environment will be integrated and
effective.
A mental representation is a process used to represent the external world internally, as it
is perceived by the sensory systems (Greca & Moreira, 2000). An associated concept can be
found in the mental image which, according to Bishop (1989), is obtained from the visual
processing of an individual combined with his or her interpretation of the figurative information
perceived. The images are, for Johnson-Laird (1983), as much a product of perception as of the
imagination, and correspond to concrete visions of a mental model created by the individual. He
also proposes that people reason through mental models which turn out to be an internal
representation of information that analogously corresponds to that which is being represented
(Johnson-Laird, 1996).
2
In understanding that a great part of our mental models are constructed from vision, and
that they are fed by other sensorial avenues, blind people need to maximize their use of the other
channels of sensory information in order to construct effective images of the environment. In this
way, it would be most logical to think that the mental models developed by blind people are less
exact in relation to those elaborated by sighted people.
In this subject, Lahav (2005) proposes that the biggest difficulty for blind people lies in
the fact that the lack of vision and subsequent impossibility to collect information by means of
the visual sense impedes their generation of effective mental maps of a determined space and the
ability to utilize them competently. She suggests that the mental mapping of spaces and of the
possible routes to navigate those spaces is essential in order to develop orientation and mobility
skills. To compensate for the lack of vision by using the other sensory channels to collect
information from the surroundings would contribute to the anticipation of mental maps and
unknown environments and would improve the spatial performance of blind people.
There has been a diversity of investigations to explore how blind children can come to
know space and navigate in a virtual environment by using sound. Mereu & Kazman (1996)
proved that 3D audio interfaces used by a blind person helps him or her to locate a given point
within a three-dimensional space. Lahav & Mioduser (2002, 2004) studied the capacity of blind
people to develop mobility orientation and spatial knowledge skills through the use of a multisensory virtual environment, finding that the visually impaired users are capable of mentally
representing virtually traveled spaces, this being the first step towards achieving better
movement.
Simonnet, Guinard & Tisseau (2006) developed a haptic interface which allows blind
users to create and simulate embarkation itineraries through the use of their own voices. This
study approached the diverse strategies utilized by blind sailors to construct their own spatial
representations when using maps, based on recent theories that point out that those sailors who
lack vision, in spite of being able to construct cognitive maps of their environment, tend to
construct egocentric and sequential images of space.
A number of studies (Sánchez & Elías, 2006, 2007; Sánchez, Lumbreras & Cernuzzi,
2001; Sánchez & Zúñiga, 2006) have demonstrated that the use of sound-based educational
software accompanied by cognitive tasks can help children with visual disabilities to develop
diverse thinking skills impacting positively in their learning.
As regards the mental modeling of space, the pioneering investigatory work done with
the AudioDoom software in 1999 (Lumbreras & Sánchez, 1999) validated the fact that
navigation in virtual environments by means of sound and without visual cues can produce an
intellectual development that allows a blind learner to easily map a determined physical space
(Sánchez et al., 2001).
However, following the promising results obtained with AudioDoom in as much as being
concretely modeled from a virtual environment, we wonder if it is not only possible for blind
learners to achieve representation of the environments traveled with a maximum level of
3
accuracy, but if they could also achieve spatial knowledge and efficient navigation by using
sound-based interfaces with distinct styles of navigation.
Research Problem and Hypothesis
Even if the mental models are constructs of an individual nature that cannot be directly
explored, they can be symbolized and represented through a physical model. Through these
representations we mean to study how blind users, using audio-based, free and restricted
navigational interfaces, are able to model space.
With this purpose in mind, it interested us to inquire into what exactly are the mental
representations of a navigationally free and restricted virtual environment, constructed by blind
or visually impaired users through audible cues.
This allowed us to define three assumptions:
1.
Mental representations from a restricted navigational environment are more exact
than those from a free environment.
2.
Mental representations from a restricted environment require less virtual travel
than mental representation from a free environment.
3.
Mental representations of restricted spaces are more extensive in as much as the
quantity of the map that is able to be represented, than is that of the free environments.
AudioDoomII Description
AudioDoomII (Sánchez & Zúñiga, 2006) is an audio-based virtual environment that
emulates the traditional game “Doom”, and that offers the visually impaired user useful auditory
cues to orient him or herself spatially within the game’s different mazes. It is an expanded and
upgraded version of AudioDoom (Lumbreras & Sánchez, 1999) including more maps and mazes
and logging actual use tools.
Mode of Interaction
In order to navigate the software, those computer keyboard keys that are most familiar for
blind users have been selected for use (see Figure 1). In this way, they use the keys f and J
which have a relief mark in all keyboards, and which serve blind users as a point of reference.
Model of the Environment
The metaphor followed by AudioDoomII is associated with the original Doom game,
where the player and protagonist must run through a maze which simulates the corridors of a
space ship, confronting distinct acoustic elements which represent monsters and other creatures,
with the objective of finding the way out and being able to save the world from these threats.
The game is composed of different levels, in which each corresponds to a distinct maze and that
4
imply higher levels of complexity.
FIGURE 1. Keys utilized for interaction with AudioDoomII (darkened keys)
For the specific use of this study, a special and more complex maze was developed (see
Figure 2), in order to challenge the auditory and spatial memory abilities present in the young
group of participants.
FIGURE 2. Image of AudioDoomII maze utilized as a virtual environment of restricted
navigation
Feedback for the User
Travel in the above mentioned maze is accessible to the blind user through audible cues.
The configuration of the sound used in the software was stereo sound. In this way each element
in the maze (monsters, mutants, doors) and each action taken by the player (walking, picking up
bullets, shooting, opening doors), possesses a characteristic sound that also provides references
about the player’s spatial location which could be left, right or center, in which the sound is
heard from both sides.
Graphic interface
The graphic interface provided by AudioDoomII can be seen in Figure 3 (for the
5
facilitator, educator or parent), in which each sound action and interaction with the elements of
the game produce a written visual feedback to the left of the screen.
FIGURE 3. AudioDoomII interface showing the written visual feedback of the actions performed
by the user
Audiolink Description
The AudioLink software (Sánchez & Elías, 2007) corresponds to a role playing game
(RPG) in which the player controls a main character through which he or she interacts with the
virtual world and its elements. The user can navigate distinct scenarios, interact with other
virtual characters, take up and use objects as well as various other actions, with the purpose of
fulfilling a series of objectives that end up tying together the different storylines presented in the
game. It is also possible to attempt optional searches which lead to additional benefits.
To increase the playability of AudioLink, all the missions have associated rewards for the
protagonist, such as distinct items or objects that allow him or her to access new searches and
rewards. Each of these missions is also associated with different science learning concepts. The
successful completion or incompletion of the different missions produce different sub-storylines
and outcomes in the game, in that the game includes sequential, parallel, optional and alternative
storylines.
Mode of interaction
The virtual interaction is performed by means of the keyboard, where those keys that are
most familiar and frequently used by the end users were selected for use (see Figure 4).
6
FIGURE 4. Keys used for interaction with AudioLink (darkened keys)
Model of the Environment
The navigation through the different environments and spaces replicates a successful
model used for commercial videogames designed for blind users which follow a third-person
metaphor with a camera view fixed from an above and behind position. This model of
navigation has been adjusted to the needs of blind children, having added auditory cues of
quadraphonic sound which allows them to form a mental construction of the space.
The representation of the surroundings is described in a XML file in which all the
attributes that characterize the different scenes can be found. In the xml structure (see Figure 5)
there are a series of elements that define the storylines, which can also be modified, extended or
eliminated through its structure.
FIGURE 5.Representation of the virtual world’s structure
Feedback for the User
The sounds are key elements in the development of AudioLink. Following the first
evaluations the need to employ spatial sound was apparent, and was done using a configuration
of quadraphonic sound which supports the construction of mental spaces by the users.
In this way, 2D sound tracks were used for information that does not require spatial
references (dialogues, scene descriptions, help and warnings), and spatial sound was used for all
the elements that are necessary to locate spatially within the game (footsteps, doors, characters
and objects).
Graphic Interface
7
In this software a large-scale virtual environment is created, incorporating more than 150
different scenarios (see Figure 6). These are grouped together in 6 characteristic zones: forest of
light, black island, lost desert, central city, white mountains, and Thor’s volcano. Each of these
places has a particular graphic and is differentiable from the other spaces.
FIGURE 6. Image of AudioLink’s central city, a free navigation virtual environment
Evaluation
Methodology
The research performed was of an exploratory-descriptive nature, with special emphasis
on the observation of the phenomenon in question and on the analysis and interpretation of the
information processed and the comparison of data. It is a preliminary study in which the results
obtained and collected will constitute the foundation in a continuing and deepening research of
the object of study in future stages.
The main search that we initiated with this research was focused on the comparison
between the mental representations that the blind participants formed a restricted virtual
environment versus a free navigation virtual environment.
More specifically it was proposed that the participants would be capable of mapping,
with concrete material, the maze simulated in the restricted virtual environment. And, in the end,
with the results of both projects there would be a comparison of the representations formed by
participants with the recordings stored in the software in order to process and analyze the data
contained in each software.
Participants
8
The sample group selected for this study consisted of 9 young people of both genders,
aged 18 to 31. All the subjects had been diagnosed with blindness or low vision, all legally blind
(see Table 1).
TABLE 1
Sample description
Case
1
2
3
4
5
6
7
8
9
Gender
M
F
M
M
F
F
M
F
F
Age
22
28
29
20
22
27
31
27
18
Diagnostic Oftalmologic
Retinitis Pigmentosa
Diabetic Retinitis
Retinitis Pigmentosa
Atrophy of the Optic Nerve
Neurofibromatosis, Neucoma
Diabetic Retinopathy
Chronic Uveitis
Retinopathy
Myopia, Retinitis Pigmentosa
Level Of Vision
Low vision
Totally Blind
Low vision
Low vision
Low vision
Totally Blind
Totally Blind
Totally Blind
Low vision
Procedure
The work was carried out during a period of three months, with one work session of 1.5
hours each week. Each of these sessions corresponded to a phase of cognitive testing (see Figure
7) which was configured on the basis of two learning activities in agreement with each software
program.
FIGURE 7. Participants during the distinct phases of the cognitive testing
In the first session both virtual environments (AudiodoomII and AudioLink) were
presented to the participants so that they could familiarize themselves with the operations and
interact with each one. In the following sessions, and to maintain the interest and motivation of
the participants, we proceeded to the alternated execution of the learning activities.
The procedure utilized in the process of the interaction between the user and the software
consisted of the running of a maximum of three continuous gaming, and posterior representation
9
of the space traveled. The duration of each run depended on the performance of each participant.
The didactic sequence in the development and progression of both the learning activities was:
1.
Concrete representation through the use of a board and Lego pieces. The learner
arranges the pieces in a surface constructing the labyrinth or path followed during gaming.
2.
Graphic representation using a whiteboard and markers. Each user draws on the
blackboard the labyrinth or path followed until reaching the destination spot.
3.
Representation by corporal movement, simulating the route taken in the game. By
the means of the learners own corporal movements each learner shaped his or her mental
registering and the extension of each corridor reached in every virtual space.
In the case of AudioDoomII, whose maze complex was composed of three corridors, to
complete a more efficient interaction, each user was requested to choose and represent only one
of the routes traveled. Afterwards, and once this objective had been accomplished, it was
possible for the users to choose another of the maze corridors and follow the same procedure to
its completion. Complementarily, records were taken from the simulation of the body
movements made by the users while they traveled in the game. In this way it was possible to
trace and represent in the most accurate way the mental record of the extension of each corridor,
each with its own corresponding movements.
In the learning activity for AudioLink, the participants were requested to map out at least
one of the possible routes from the start of the game, that is, from the house of the main
character, to the end point, which is the house of the secondary character. In this way, the
participants had to make an exhaustive exploration of the virtual surroundings in order to
complete the task.
Instruments of Evaluation
For the analysis and evaluation of the interactions and virtual navigation carried out by
each user in every one of the software programs, the event recording tools and data processing
methods described below were used.
AudioGram
AudioGram is a tool that helps in the analysis of quantitative data obtained during the
interaction between a user and the virtual world. This tool is capable of taking the players’ login
file and displaying an analysis of any particular game’s information on the screen. In the analysis
a time-line containing different events representing the player’s actions in the game is shown.
The actions are, for example, the opening of doors, shots fired, reloading bullets, the elimination
of monsters and steps taken among others. It is also possible to move along the time-line in both
directions in order to observe some particular sequence of actions in more detail. This option is
associated with two additional elements: a textual description of the actions, the same as that
which is presented during the game and a miniature map that shows the exact position of the
player on the map at the selected time (see Figure 8).
10
Using this display as provided by AudioGram, it was possible to observe and keep
evidence of the number of virtual routes that each user traveled for each of the halls in the maze.
AudioGram also provides the possibility to observe some variables in the long-term, such
as the total game time, the quantity of actions, different errors and encounters among others.
This data is depicted by the software in order to show the gaming variations of a single subject.
In the horizontal axis of the graphic, the number representing each particular game appears, and
the program gives the option to select a specific game in order to open a detailed analysis in
another window (see Figure 8).
FIGURE 8. Audiogram displaying the analysis of one and various games
In figure 8 on the left, the analysis and the information on a single game session offered
by Audiogram and stored in the software can be seen. On the right, a graphic corresponding to a
series of interactions performed by the same user in distinct game sessions is presented.
LinkGram
LinkGram is a support tool for the use of AudioLink educational software, designed to
process and analyze the data of a single game session that has been recorded in the software.
11
FIGURE 9. Linkgram interface
When the user loads this program, a first window is launched with information relating to
the scenarios traveled by the player, the times that the player stays in each scenario, the name of
the scenario, the total game time, an inventory of the objects collected by the user and an option
to revise a more detailed account of any particular scenario. For this last one, a second window is
generated which is illustrated on the right in figure 9, where a map of the game appears in the
upper right hand of the screen, as well as a list of the actions performed and the general statistics
with respect to the movements made in the scenario under review, such as the number of
encounters, the number of objects obtained, both used and unused, conversations with other
characters, and a request for descriptions of the most important alternatives for the analysis.
Guideline for evaluation of the exactitude of the representations
To evaluate the mental model that each learner could shape in his or her concrete
representations, a measurement instrument was designed by the authors. Two special education
teachers that are specialists in children with visual disabilities administered the initial and final
evaluation as well as the cognitive testing.
Initially the evaluation instrument was designed based on a series of 22 indicators related
to degree of exactitude of the mock-ups made by users. However, from this initial instrument we
detached a second and final measurement instrument that classifies and includes the initial 22
indicators into the following 5 essential indicators:
1.
Adequate Location: the correct position within the concrete representation,
provided to the starting and final spots of the route.
2.
Shows landmarks: the inclusion in the map of elements such as objects, people,
and sounds that the learner confers high relevance and uses as aids to map the path followed.
3.
Proportional use of the halls and paths: similar or balanced dimensions in the
length and width used by the learner to replicate the routes.
4.
The routes taken are of a correct orientation and direction: each path or corridor
is designed in congruence with the virtual space navigated.
12
5.
The model is complete and precise: referred to the extension or totality of the
represented map and the degree of similitude or fidelity with the topography of the virtual space
navigated.
Each representation made up by the users was examined according to these criteria,
which allowed determining an index of achievements for each of the reproductions.
The scoring for each design fluctuated between 0 points, for a minimal grade of
elaboration, and 5 points for the maximum grade of exactitude according to the previously
mentioned criteria. Through this data a general performance average was obtained for each
participant in the representations elaborated for both virtual environments separately.
Results
The results obtained are promising. In total, the sample group was able to elaborate 45
representations in the restricted navigation environment (AudiodoomII), and 31 representations
of the free navigation environment (AudioLink)
From the data displayed it is possible to determine that for a restricted environment like
AudiodoomII, on average, 15 interactions with the software are required per participant in order
to form 5 representations. For an open navigation like AudioLink, on average, 9 interactions with
the software are required per participant in order to form 3 representations. In both cases, a
relation of 1 to 3 is obtained in the number of runs to the number of reproductions.
TABLE 2
Average number of virtual runs and representations performed by each participant
Sample
Average
Virtual
Runs
AudioDoomII
AudioLink
Number of
Representations
AudioDoomII
AudioLink
15.1
8.8
5.2
3.4
According to the representations elaborated by the sample group, the following is a
presentation of the average level of exactitude achieved by each participant in the reproduction
of both virtual environments. It is worth pointing out that each participant formed a different
number of reproductions.
Judging from the averages obtained in the evaluations of both software it is possible to
indicate that, as for representations, AudioLink has slightly higher exactitude scores than
AudiodoomII, scoring 6 decimals of difference even when, according to the information
displayed above (see Table 2), the number of representations formed in AudioLink are inferior
(3) to those formed in AudiodoomII (5).
13
FIGURE 10. Average of exactitude achieved in the representations of both virtual environments
In as much as the exactitude scores of the two environments, 6 of the participants
obtained the best scores in their representations for AudioLink and 3 participants scored as high
in AudiodoomII (see Figure 10).
Analysis of the Results according to the Evaluation Guideline
The following is a presentation of the results obtained for each indicator included in the
guideline for the evaluation of exactitude utilized to determine the degree of accuracy of the free
navigation representations (AudioLink) and those of restricted navigation (AudioDoomII).
Indicator 1: Appropriate location of the starting and finishing points. For AudioLink,
27 of the 31 representations elaborated by the sample group (87%) obtained favorable scores
with respect to a better location of the starting and finishing points in a given run. In
AudioDoomII, 24 of the 45 representations of the maze (53%) obtained optimal results (see
Figure 11).
Indicator 2: Shows signs or points of reference within the map.
In AudioLink, 25 of the 31 representations (80%) incorporated landmarks. In
AudioDoomII, 26 of the 45 representations (57%) incorporated supporting references within the
map (see Figure 11).
Indicator 3: Proportional use of the corridors or paths is observed. For AudioLink, 1 of
the 31 designs (3.2%) shows proportionality. This lone case, however, is one which obtained the
maximum score for this indicator. In AudioDoomII, 16 of the 45 designs (35.5%) show higher
proportionality in their maps (see Figure 11).
Indicator 4: The routes represent the correct orientation and direction. For AudioLink,
18 of the 31 representations were favorably evaluated, which corresponds to 58% of the sample
group. In AudioDoomII, 24 of the 45 representations were favorably evaluated, corresponding to
77% (see Figure 11).
The previous indicators 3 and 4 show AudioLink with lower indexes than AudiodoomII
14
for its representations. In these cases, the conditions of free navigation software like AudioLink
allow the participants to move in any direction, as long as they get around the obstacles that are
presented in their path. This property of the game is that which makes a difference with
AudiodoomII, where each participant has only one alternative or way to accomplish a
predetermined route, facilitating the structuring of that virtual space.
Indicator 5: The model is complete and precise. In order to determine the exactitude of
the reproductions, we considered it necessary to explore in depth the criteria that are considered
as the desirable qualities in this item. By “complete model” we refer to the extension or totality
of the map that is represented and “precise model” refers to the degree of similarity and accuracy
with the topography of the virtual space traveled.
For AudioLink, 10 of the 31 representations (32%) are considered to be complete and
precise. In AudioDoomII, 17 of the 45 representations (37%) are observed as complete and
precise (see Figure 11).
In this regard, the percentage is slightly higher in AudiodoomII. However, using the
representation it is possible to observe that while the number of interactions increases in one
software, or the other, the maps created also increases their levels of precision and extension.
FIGURE 11. Comparison of the results in scores obtained for the representations of both
software programs
It can be established, then, that the degree of exactitude in the representations of
AudiodoomII achieved a total result of 59%, while the general exactitude for AudioLink reached
45%.
Conclusions
Having presented the results of this study and in agreement with the performances of the
sample users in the elaboration of concrete representations of the virtual environments traveled,
it is possible to conclude that the mental representations of a closed environment are more exact
that those of an open environment.
15
Based on the results obtained it is possible to sustain the assumption: Mental
representations from a restricted navigational environment are more exact than those from a free
environment. The evaluation of exactitude expressly demonstrates that the representations
formed on the basis of the restricted navigation environment (AudioDoomII) are more precise,
reaching a total percentage of 59% and scoring 14 percentage points of difference with respect to
the results obtained by the navigation software operating in an free environment (AudioLink),
which averaged 45% exactitude.
Even though AudioDoomII representations showed higher fidelity with the virtual
environment, the representations designed for free navigation in AudioLink were not far away
from the originals, rather, they surprisingly reached progressive levels of exactitude.
The sound quality (audio cues) was a determining factor to form more exact mental
representations in a free environment. AudioDoomII provides concrete audible cues in the way of
footsteps when walking/advancing, which permits the user to be able to estimate the dimensions
of the corridors traveled and, in this way, to more easily represent them afterwards.
AudioLink, in spite of the fact that it too provides auditory cues that inform the use on the
spatial progress of the character, does not use a system that is equally tangible, which explains
the difficulty demonstrated by the group of participants to model this space with high levels of
exactitude.
The presence of a diversity of objects and concrete audio cues in the virtual environments
allow to conjecture that learners with visual disabilities require points of reference to represent
free environments with higher fidelity.
The indicator in which AudioLink reaches the highest percentage of achievement is
precisely that which refers to the depiction and use of landmarks within the map.
In this way, and judging from the evidence offered by the evaluations of exactitude, we
can establish that the blind participants require points of reference and very concrete auditory
keys to be able to represent open environments with higher degrees of accuracy.
As such, we can conclude that the mental representation of a restricted environment
requires less virtual travel than the mental representation of a free environment. If, as it is, this
hypothesis ends up being effective, there are also many considerations.
In a free navigation environment there are differences associated with the act of
exploring the space, given that it allows for multiple movements in any direction and is
composed of numerous characters and elements with which it is possible to interact and obtain
information, which makes navigation in this kind of space always slower in comparison to the
other software.
In a restricted navigation environment, although it is configured on the basis of a maze
complex, the participants have to choose one of three routes, each one with a distinct dimension,
in order to form a concrete representation. If the dimension was different in each maze, the
16
necessary time used to explore each one would also be quite different.
In AudioDoomII, the participants of the sample group achieved an average of 15 virtual
runs in the game, while in AudioLink the average number of runs is 8 (see Figure 11) for each
member of the sample group. Therefore, it is possible to affirm that AudioDoomII favored more
precise interactions between the user and the software in as much as quantity is concerned, but
this is not so in as much as the time used in the navigation. On the contrary, the characteristics of
AudioLink needed more extensive interactions in as much as the time, and less when referring to
the quantity of interactions with the environment.
With AudioDoomII the participants required approximately 6 virtual runs in order to form
a more exact representation; with AudioLink they had to explore the space more than 6 times in
order to achieve an equally effective representation.
As such, in that they are less extensive, the interactions with AudioDoomII were briefer
as far as time is concerned, which allowed each user a higher number of runs in one single
session. On the other hand, the interactions with AudioLink were slower, and as such each
participant needed to make more detailed explorations of the environment.
As a consequence, less runs and less interaction time was necessary in the navigation of
the restricted virtual environment (AudioDoomII). However, on the basis of this information it is
important to point out that this was possible due to the characteristics of the maze traveled, as
opposed to the characteristics of the free environment.
These results also do not permit us to conclude that the mental representation of restricted
spaces is more extensive as far as the quantity of the map that is able to be represented than that
of the free environments. This assumption has ended up to be inexact, as through this research it
was possible to prove that the blind users are capable of representing both restricted and free
environments.
Of the total amount of representations created for AudioDoomII (45 designs), 76%
corresponded to a complete version of the maze. In the representations elaborated for AudioLink
(31 designs), 90% of them achieved a reproduction of the total extension of the space. As such,
7 of the 9 participants of the sample group were capable of elaborating complete representations
of a free environment.
We want to emphasize participant 1, who was not only one of the participants who
achieved the tasks in the most conscious and dedicated manner, but who was also the only
participant who was able to precisely represent the three routes of the maze in AudioDoomII.
Only one participant, and one that has total congenital blindness as opposed to those who have
acquired blindness, did not achieve a positive result in any of the environments traveled.
At the beginning of this report we asked ourselves, what exactly are the mental
representations of a free or restricted virtual environment, that blind users or those with visual
impairment are able to construct by using auditory tracks?. Judging from the results obtained it is
possible to sustain that the spatial image that blind participants are able to represent after having
17
explored a sound-based virtual environment with characteristics of restricted navigation as with
AudioDoomII, is more exact when compared to the representations formed for the world of free
navigation in AudioLink.
This reveals that the mental model achieved by the blind participants in a free navigation
environment like AudioLink, results in less exact representations of the surroundings. Even when
observed as being completed and with increasing degrees of accuracy in representing the virtual
space, they cannot be modeled with all the details and are designed with a low level of
proportionality when compared to the representations elaborated for AudioDoomII.
It is possible to confirm that the auditory cues integrated into both software programs
allow for and favor the construction of a mental model of space. This verifies that the more
concrete and direct the auditory cues (footsteps in AudioDoomII), the higher the levels of
exactitude when modeling the space. To the contrary, when these auditory cues are less
concretely perceived, the representation will be less fitting to the virtual world and will lose
degrees of exactitude.
The participants of the sample group, the majority of them with low levels of vision, were
able to elaborate representations that approach an average of 3 points of exactitude, out of a
maximum of 5 points.
However, it is possible to observe that the only participant that had total congenital
blindness did not achieve positive results in any of the two environments traveled. That is to say
that the explorations made in both virtual spaces were insufficient to achieve the development of
an efficient spatial image of the virtual surroundings, be it a restricted or a free environment.
Discussion
In the light of the results presented we can reflect more deeply not only on the capability
of users with visual disabilities to elaborate mental representations that replicate with high
fidelity the virtual space navigated, but also on the diversity of digital tools that are designed and
provided to these users.
The fact that learners can explore and comprehend a virtual environment and make a
concrete replication refers directly to the capacity to concretize the abstract through audio cues
that conveys clearly the real and specific information about what is interacted with. This
situation projected to the everyday life in the school is proposing the use of a quality audition
based on quality sound emissions able to convey to the person with visual disabilities a mental
image in accordance with the real world, that is, with concrete physical characteristics such as
height, depth, width, and length. This opens a wide array of new opportunities for those who lack
of functional vision, such as a scenario were they could walk with the aid of a technology that
allow them to “see” the world through sound.
The results obtained with young and adult participants interacting with restricted and free
navigation environments highlight the question of whether these performances can be similar if
we use virtual simulations of free and restricted real environments. For example, if it can be
18
possible that a group with similar characteristics would be able to identify a simulated real life
environment using a software program, would it be possible that they could differentiate,
compare or even prove distinct routes or paths taken in a virtual environment with others taken
on a day-to-day basis in a real life context?
After this experience, we can imagine a scenario such as a participant interacting with the
virtual maze software (AudioDoomII), then being asked to run two or three circuits of real life
corridors so that he or she could compare and recognize which of them is identical to that which
he or she explored virtually.
The exploration of new alternatives in the use of spatial sound, using 2D, 3D, and
quadraphonic sound seems to be a key element. Another contribution would be to integrate and
complement the auditory experience, for example, with the use of other technological tools such
as haptic interfaces that optimize and bend back over the sensory and cognitive experience.
Acknowledgements
This report was funded by the Chilean National Fund of Science and Technology,
Fondecyt, Project 1060797
REFERENCE LIST
Bishop, A. (1989). Review of Research on Visualization in Mathematics Education.
Focus on Learning Problems in Mathematics. 11 (1-2), 7-16.
Carreiras, M., & Codina, B. (1994). Internal spatial representation of the blind from
verbal descriptions. Foundation of Infants and Learning. 6 (2), 145-158.
Greca, I., & Moreira, M. (2000). Mental Models, Conceptual Models, and Modeling.
International Journal of Science Education. 22 (1), 1-11.
Johnson-Laird, P. (1983). Mental models: towards a cognitive science of language,
inference, and consciousness. Cognitive Science Series, 6. Cambridge, Mass: Harvard
University Press.
Johnson-Laird, P. (1996). Images, Models, and Propositional Representations. In M. de
Vega (Ed.), Models of Visuospatial Cognition (pp. 90-126). New York: Oxford University Press.
Lahav O. (2006). Using virtual environment to improve spatial perception by people who
are blind. Cyberpsychology & Behavior: the Impact of the Internet, Multimedia and Virtual
Reality on Behavior and Society. 9 (2), 174-7
Lahav, O., & Mioduser, D. (2002). Multisensory virtual environment for supporting blind
persons’ acquisition of spatial cognitive mapping, orientation, and mobility skills. In P. Sharkey,
C. Sik Lányi, & P. Standen (Eds.), Proceedings of 4th International Conference Series on
Disability, Virtual Reality and Associated Technologies (ICDVRAT’02; pp. 213-220). UK:
University of Reading.
19
Lahav, O., & Mioduser, D. (2004) Blind Persons’ Acquisition of Spatial Cognitive
Mapping and Orientation Skills Supported by Virtual Environment. In P. Sharkey, R.
McCrindle, & D. Brown (Eds.), Proceedings of 5th International Conference Series on
Disability, Virtual Reality and Associated Technologies (ICDVRAT’04; pp. 131-138). UK:
University of Reading.
Lumbreras, M., & Sánchez, J. (1999). Interactive 3D sound hyperstories for blind
children. In M. Williams, M. Altom, K. Ehrlich, & W. Newman (Eds.), Proceedings of ACM
CHI 1999 (CHI'99; pp. 318-325). USA: ACM Press
Mereu, S., & Kazman, R. (1996). Audio-enhanced 3D interfaces for visually impaired
users. In M. Tauber (Ed.), Proceedings of ACM CHI 1996 (CHI’96; pp. 72-78). USA: ACM
Press
Sánchez, J., & Elías, M. (2006). Blind Children Learning Science through Audio-Based
Interactive Software. In M. Redondo, C. Bravo, & M. Ortega (Eds.), Proceedings of VII
International Congress on Human-Computer Interaction (Interacción’06; pp. 591-600). Spain:
Lince Graphic Arts
Sánchez, J., & Elías, M. (2007). Science Learning by Blind Children through AudioBased Interactive Software. Annual Review of CyberTherapy and Telemedicine: Transforming
Healthcare through Technology, 5, 184-190
Sánchez, J., Lumbreras, M. & Cernuzzi, L. (2001). Interactive virtual acoustic
environments for blind children: Computing, Usability, and Cognition. In J. Jacko, & A. Sears
(Eds.), Proceedings of ACM CHI 2001 (CHI’01; pp. 65-66). USA: ACM Press
Sánchez, J., & Zuñiga, M. (2006). Evaluating the Interaction of Blind Learners with
Audio-Based Virtual Environments. Annual Review of CyberTherapy and Telemedicine. Virtual
Healing: Designing Reality, 4, 167-173.
Simonnet, M., Guinard, J., & Tisseau, J. (2006). Preliminary work for vocal and haptic
navigation software for blind sailors (2006). In P. Sharkey, T. Brooks, & S. Cobb (Eds.),
Proceedings of 6th International Conference Series on Disability, Virtual Reality and Associated
Technologies (ICDVRAT’06; pp. 255-262). UK: University of Reading
Ungar, S., Blades, M., & Spencer, C. (1996). The construction of cognitive maps by
children with visual impairments. GeoJournal Library. 32, 247.
20