YouDash3D: Exploring Stereoscopic 3D Gaming
for 3D Movie Theaters
Jonas Schild**a, Sven Seeleb, Maic Masucha
Entertainment Computing Group, Univ. of Duisburg-Essen, 47057 Duisburg, Germany;
b
Bonn-Rhine-Sieg University of Applied Sciences, 53757 Sankt Augustin, Germany
a
ABSTRACT
Along with the success of the digitally revived stereoscopic cinema, events beyond 3D movies become attractive for
movie theater operators, i.e. interactive 3D games. In this paper, we present a case that explores possible challenges and
solutions for interactive 3D games to be played by a movie theater audience. We analyze the setting and showcase
current issues related to lighting and interaction. Our second focus is to provide gameplay mechanics that make special
use of stereoscopy, especially depth-based game design. Based on these results, we present YouDash3D, a game
prototype that explores public stereoscopic gameplay in a reduced kiosk setup. It features live 3D HD video stream of a
professional stereo camera rig rendered in a real-time game scene. We use the effect to place the stereoscopic effigies of
players into the digital game. The game showcases how stereoscopic vision can provide for a novel depth-based game
mechanic. Projected trigger zones and distributed clusters of the audience video allow for easy adaptation to larger
audiences and 3D movie theater gaming.
Keywords: S3D gaming, stereoscopic 3d vision, interactive cinema, game design, movie theatre, mixed reality,
audience participation, crowd gaming
1. INTRODUCTION
In the recent years, many cinemas have been equipped with digital projection technologies, often offering stereoscopic
projection for 3D movies. Since the tremendous success of James Cameron's Avatar, expectations for 3D content are
high. Besides movies, other 3D content could offer additional value to the increased admission of 3D digital projections,
e.g. interactive entertainment. Before a movie starts, the audience normally watches commercials. Many people use this
time frame for communication and for getting settled while waiting for the film to start.
Our approach was to use this time period for an interactive 3D event that offers exciting entertainment to the audience.
We thus aimed at creating a game for being played by a movie theater audience in stereoscopic vision mode.
Like movies, digital games have recently promoted stereoscopic vision as a strong differentiator. The Playstation 3 and
the Nintendo 3DS offer stereoscopic 3D (S3D) gaming on consoles while driver-based solutions such as Nvidia 3D
Vision or TriDef let the user play in S3D vision on the PC. In all these scenarios, existing games or well-known game
concepts are simply extended with a S3D vision mode as a toggle option. These games usually hardly deviate from their
planar counterparts. So far, none of them utilize stereoscopy as mandatory gameplay element [1].
In our work, we thus aimed at two goals: (1) providing interactive entertainment in a 3D movie theater setting; (2)
offering gameplay mechanics that make special use of stereoscopy. These goals yield problems that stretch across
several fields. The next section (Section 2) presents the current state of stereoscopic gaming. Section 3 then explores the
actual situation of a gaming event in a movie theater, namely interaction possibilities and lighting. Based on our
findings, we develop game mechanics that provide depth-based gameplay in a flexible and casual game setting (Section
4). This leads to our prototype YouDash (Section 5), which showcases our concept in a reduced kiosk setting.
* [email protected]; phone +49-203-3791652; http://medieninformatik.uni-due.de
2. STEREOSCOPIC GAMING
The idea to provide stereoscopic video games is not a recent one. It has been used in several peripheral devices for video
game consoles, such as the 3D Imager for Vectrex (GCE, 1982), the SegaScope 3D (Sega, 1988) for the Master System,
or even a dedicated 3D video games console such as the Nintendo Virtual Boy (Nintendo, 1995) [1,2]. None of these
systems were commercially successful [1], possibly due to technological constraints, lack of quality content and other
factors like marketing strategies.
With the re-emerging technological trend of stereoscopy in digital movies [3], game companies have released several
systems offering S3D effects in games, i.e. the Playstation 3, the Nintendo 3DS, or Nvidia 3D Vision offer S3D vision as
an option for existing games. These often provide identical game mechanics but presented with S3D graphics. Only little
impact on gameplay is known to result from S3D vision in games. Rajae-Joordens evaluated gameplay of Quake III:
Arena on an autostereoscopic display for 2D vs. S3D display modes. 85% of the participants preferred S3D over 2D.
The gaming performance (score) initially was higher in S3D but equal to 2D after prolonged use [4]. LaViola and
Litwiller also investigated whether user performance benefits from 3D stereo in digital games. Their results indicate that
although participants preferred playing in S3D for the tested games, it would not provide any significant advantage in
overall user performance. In addition, users' learning rates were comparable in the 3D stereo display and 2D display
cases [5].
Compared to these findings, gameplay could benefit from stereoscopic presentation on different levels. Previous work
suggests that stereoscopic depth cues can provide more precise information for distance or motion judgment than
monocular cues. Consequently, players could perform in-game tasks more precisely and more efficiently [6,7]. In a
recently submitted paper, we presented a comprehensive list of stereoscopic game design ideas and concepts which we
extracted from identified cognitive limits and special properties of human vision and technical systems [8].
Based on these assumptions, we considered depth perception as primary subject for designing game mechanics. Such
game mechanics could involve depth-estimation tasks through consciously misbalancing depth cues e.g. texture gradient,
lighting, shadows, transparency, or relative position, solvable through S3D. But how can such depth-based gameplay be
designed effectively in an application? As we will see in the next section, the setting and the play environment, i.e. a 3D
movie theater, have significant impact on game design.
3. 3D MOVIE THEATER SETTING
Interactive play in a movie theater setting requires a comprehensive technical setup. One key parameter of such a setup
is how the audience is supposed to interact with a game. Such interaction considerably differs from personal gaming on a
video game console as it involves a large group of players interacting on the same screen.
3.1 Previous Movie Theater Games
The potential for audience participation has been acknowledged many times before. With their audience movement
tracking, Maynes-Aminzade et al. [9] could detect whether the audience leaned to their left or to their right. In another
approach, laser pointers were distributed to the audience. The projected laser dots on the screen were tracked to select
certain areas on the display. These methods worked as input for a derivative of the “Whack-A-Mole” game, steering a
race car, quizzes, or similar applications. In all examples, only the audience as a whole is able to participate. It is
impossible for individual members to determine their influence on the input. In some cases, portions of the audience
were not even recorded, but still participated fully.
An earlier development, the Cinematrix Interactive Entertainment System [10,11], used reflective paddles distributed to
the audience to determine application input. The paddles each had a red and a green side, which could be used by the
audience to participate in on-screen events like opinion polling or maze navigation [9]. While this approach is able to
identify each audience member by means of the input devices, it is impossible for the people in the audience to establish
this connection. One solution would be to give each player of the audience a respective representation on the screen.
In another popular case, Maynes-Aminzade et al. tracked the on-screen shadow of a beach ball and used it for input [9].
Consequently, as soon as individual people batted the ball, they became immediately aware of their own actions in the
game. This approach had two disadvantages. Firstly, only a single member of the audience could participate at a given
point in time, leaving the rest waiting for their turn. Adding more balls would make it impossible to track which shadow
belonged to which ball. Also, such a solution would require additional equipment for the audience, such as controllers.
Distributing controllers among the audience provides multidimensional input at the price of additional costs. On the
other hand, today, controller-less gameplay offering direct interaction is common in many casual games. In addition, the
entire interaction could be confined to a single portion of the audience, e.g. to the front rows, if this portion refused to
pass the ball backwards. This could exclude large parts of the audience.
Recent commercial cinema games have used these approaches for advertisement events. NewsBreaker allows an
audience to control a breakout game through crowd-based body movement. It was installed in five movie theaters in the
United States in 2007 [12]. Another game for Volvo allowed the audience to steer a car on a racing track [13]. Asteroid
Storm by iO, an advertisement game for O2 in 2009, was the first interactive cinema game featuring stereoscopic 3D. As
with other current S3D games, the gameplay did not make any use of the 3D effect besides for a distinguished
presentation. Again, interaction was provided by the audience through having the majority lean to the left or to the right
to trigger directional decisions at certain points of the game story [14].
3.2 Audience representation
Despite the aforementioned disadvantages (e.g. limited interaction using averaged input) the demonstrations of these
systems were very successful, according to the respective authors. However, this kind of event has not been established
in today's movie theaters yet.
Thus, we believe that a combination of collaborative group activities and the inclusion of individual, identifiable
interactions, could improve these experiences. Displaying the audience as part of the game environment would already
help the players to establish a connection between their real-world and in-game actions.
This idea has been explored by Sieber et al. in a similar setting [15]. BallBouncer lets the audience see itself on the
screen display, as in a mirror. The filmed video is augmented with virtual balls. The players' movements are being
tracked through means of image processing. By waving their hands, the players can bounce the balls towards other
members of the audience. The game simulates depth within the virtual environment according to the height of the seats
on screen. Despite these similarities, the game does not involve stereoscopic vision nor does it utilize depth information
of the scene.
Another aspect is the social situation of the movie theater setting. The aforementioned systems were mainly
demonstrated in controlled situations with known numbers of people. A 3D movie theater game should offer gameplay
for all scales of audiences and dynamic behavior: Not everyone is seated before the movie starts and certain players
could be unwilling to participate. Others might join later on or suddenly leave play to visit the restrooms. Furthermore,
singling out individual people could make them feel uncomfortable as the attention of large parts of the audience would
be directed to them. Other, more active, users might feel highly motivated to beat the current highscore and to enter the
“hall of fame”.
Figure 1. Movie Theater Setup. The audience (left) is filmed using a stereoscopic camera-rig (middle).
The image on the screen shows the virtual scene as a composite of the mirrored audience
augmented with virtual objects (right).
3.3 Technical Setting
In our approach for 3D movie theater gaming, we wanted to increase personal involvement by offering individual
participation through real-time stereoscopic representation of the audience. We therefore planned to put a 3D camera rig
in front of the audience and to show the video live on the cinema screen. The filmed scene would then be augmented
with a virtual game scene rendered in a common game engine (see Figure 1).
We tested this setting using a professional 3D camera rig with two FullHD cameras and different light sensitive optics.
The testing took place in a small but professionally equipped movie theater room of about 70 seats. The tested 3D
projection used RealD passive polarized light technology. We used Infitec/Dolby3D glasses for a first impression of
light reflection. The room offered detailed lighting settings using cold light (available in most movie theaters) and bright
spotlights, placed at the sides above the audience. We adjusted the lights to the brightest possible level in which we
subjectively perceived sufficient contrast on the screen.
As we show in Figure 2, we could not find any light setting which offered reasonable brightness in the filmed image
while allowing sufficient image quality of the projection at the same time. Within the cinema, the image is even
perceived darker due to loss of light through polarization filters or shutters. Based on our results, we also question the
quality of previous installations, namely the 3D presentation of Asteroid Storm, where the video shows a very brightly lit
audience. While our best results were achieved using spotlights, we experienced extreme light reflections within both
types of glasses. We therefore doubt that the shown system can provide for reasonable quality in a typical movie theater
setting. The images are sufficient for basic recognitions of movements, but current technology is unable to provide a
high-quality picture of the audience in 3D.
In conclusion, our testing showed significant shortcomings of current technology, mostly related to camera light
sensitivity, stereoscopic projection technology and lighting. So far, we could not solve these issues. Despite these results,
we explored possible game mechanics for stereoscopic movie theater gameplay.
Figure 2. Lighting test results using a Fraunhofer MicroHD stereo camera rig in a 3D movie theater
room at different optics and lighting settings. (1) 9 mm, 1.4f, 50% cold light, 12 dB gain, (2) same
as (1) using Adobe Photoshop auto-contrast, (3) 9 mm, 1.4f, Spotlights, 12 dB gain, (4) 4.8mm,
1.8f, 50% cold light, 18 dB gain. Good brightness and contrast can only be reached by losing
image quality caused by increased gain. Such distortions are especially painful in stereoscopic
vision and can destroy binocular fusion.
4. 3D MOVIE THEATER GAME DESIGN
A movie theater game can be played in a fixed event setting with a predefined audience, or in a flexible movie-goer
setting, where people can join and leave at any given time. As stated before, a good game should aim for the latter
setting, as this resembles the actual situation in real cinemas. Hence, the game must not rely on static audience
configurations. Previous games solved this problem through simplicity. The input values are mostly averaged across the
whole audience or groups. And the input itself only offers a maximum of two levels of interaction or voting for a certain
position on the screen. This can be achieved with or without controllers, which saves additional costs and which is
recommended for casual spontaneous play. Similarly, visual feedback of the input can be given to the whole audience, to
group input or to individual actions. Only BallBouncer (c.f. Section 3.2) offered individual impact, as the balls could
reach everyone in the audience. This feature was strongly supported by providing visual feedback of each user's actions
through showing a mirrored video of the audience on the big screen. Representation and game graphics might be offered
in monoscopic or stereoscopic vision. But how can stereoscopic vision have significant impact on the gameplay? And
even more important to movie theater owners: does it offer additional business opportunities, e.g. through advertisement
or product placement? These considerations were compiled into the following seven goals:
A 3D movie theater game should:
G1) enable product placement or branding through a variable scenario,
G2) support dynamic audience configurations, where people can join and leave at any given time,
G3) offer simple and casual gameplay without controllers for diverse audiences,
G4) allow individual interactions,
G5) provide individual feedback, e.g. through video mirroring,
G6) show stereoscopic video to utilize the current state of technology,
G7) and create a benefit out of stereoscopic gaming through depth-based game design.
4.1 Developing a S3D Game Concept based on Depth-based Game Mechanics
Stereoscopic vision offers novel gaming possibilities related to depth perception. We therefore suggest creating the main
game conflict around a depth comparison task: Are two objects located in a similar depth range? The player has to
interact when he or she estimates two objects to be at the same depth. Only if the judgment is correct, the player or
his/her group receives a benefit towards a common goal.
During game creation we looked into many other popular games and searched for a possible adaptation of our game
mechanic. Our main inspiration came from the success of the Guitar Hero (RedOctane, 2005-2009; Activision 20062010) franchise, widely considered as a „party game“ that attracts a large variety of people, even those who would not
typically play video games. In these games, a combination of auditory and visual cues instructs the player when to issue
certain input commands which simulate music instruments. The auditory cue is provided by a song, which the player has
the illusion of performing during play. The other cue comes in form of a moving track on which buttons are placed,
representing the notes to be played, when they reach a certain trigger zone. We replaced these two cues with our depthcomparison task for judging the relative depth between two elements.
While this task can be performed on a two-dimensional display by evaluating monoscopic depth cues, our previous
analysis indicate that it can be done more efficiently and more precisely using binocular vision [8]. To simplify the
connection between input and in-game reaction for the audience, the trigger zone was determined to be represented by
the audience itself. This means, the players were filmed stereoscopically and their images were placed as elements in the
game's scene. The comparable objects (formerly notes) would then approach these images. Once they reached the depth
at which the players were placed in the scene, they would need to be triggered by player actions.
Figure 3. An early sketch of the game prototype with the audience split into two teams, with a runner
assigned to each team.
In our first concept (see Figure 3), the audience was divided into two groups and placed on the left and right side of the
scene respectively. Between their images, a race track was to be laid out on which two runners would race against each
other. Three-dimensional objects would fly towards members of the virtual audience. Members of the audience should
destroy an object by moving physically, once it had reached the same depth. Performing the action on time, or likewise
at correct depth, would temporarily increase the speed of the runner belonging to that team. Not triggering the object on
time would slow that team's runner down. In both cases, the depth estimation conflict is being updated, effectively
changing depth parameters of the objects to be compared with each other. This keeps the game loop interesting.
Later on, we changed this setup to improve our game loop based on depth-related mechanics. The runners were replaced
with images of the two groups, effectively improving both player identification and depth-based conflict: Each time, the
players destroy an object at correct depth, the players image moves in depth approaching the goal. Thus, each iteration of
moving for a new object effectively bears a new depth estimation task, forcing comparison of flying objects with the
player video at a new depth layer.
This game mechanic, demonstrated in our prototype in Section 5, allows depth-based gaming that utilizes stereoscopic
vision in gameplay (G7) and in presentation (G6). Especially showing the audience in stereo is of great effect and gives
feedback to individual actions (G5). The scenario of winning a race by hitting objects at correct depths should be easily
applicable to different scenarios involving arbitrary objects to allow for product placement and advertisement (G1).
There are only two distinct procedures players can perform to interact with the game: to perform movement in order to
hit an object and to consciously avoid movement to prevent hitting objects at false depths. This is both simple and
manageable for casual players without the need to use controllers (G3).
4.2 Dynamic Audience Configurations using Trigger Zones
The only two goals left are to allow for individual interactions (G4) while supporting dynamic audience configurations
(G2), where people can join and leave at any given time.
The current setup involved objects flying towards a certain location of the audience representation. Whoever sits at that
location is asked to interact accordingly. For a trained player, the moment of interaction would be reduced to when the
object is near to the player in depth. Even worse, the others would have to watch until an object would fly towards their
location.
One possible solution to this issue is to offer trigger zones for interaction. These trigger zones are 2D shapes projected
on the audience video, according to the two dimensional position of the approaching object. Instead of letting the
audience only perform movements directly on an object at the same depth, they can induce movements to the objects'
trigger zones. As this would induce immediate destruction of an object at the wrong depth, we added an energy
parameter to each object. The task now is extended to let the object explode at correct depth, while its power can be
reduced before by manipulating the trigger zone.
Since trigger zones can be placed anywhere on the audience video, they allow sufficient control according to dynamic
changes in the audience. By moving the trigger zones across the audience, all members can be involved in the game.
Shape and size of the trigger zones can either task individual players or small clusters of people to induce damage to an
object. This would allow the players to identify their action to a certain extend while not spotlighting single members of
the audience. Still they have to coordinate with their group members to achieve optimal object destruction at the correct
depth.
5. THE GAME: YOUDASH3D
5.1 Reduced 3D Kiosk Setup
Due to the negative results of the lighting tests in a movie theater room (see Section 3.3) we decided to explore our game
concept at a reduced scale. To still apply for a public play situation that allows people to join or leave the game on an adhoc basis, we aimed at a kiosk setup for a public event. Our kiosk setup allows two people to play while standing in front
of a 3D screen, filmed by a 3D camera (see Figure 1, cf. Figure 1).
Stereoscopic display is achieved through Nvidia 3D Vision active shutter glasses and 120Hz LCD displays. The driver
hooks into Direct3D-rendering to double the draw calls of a 3D scene according to certain game-specific heuristics [16].
The driver does not support quad-buffered stereo in DirectX rendering on consumer graphics cards. We thus use a multipass stereoscopic video rendering approach (MSVR) [17,18]. It allows rendering of separately rendered scenes into one
video surface and thus enables DirectX-based rendering of S3D video within game engines on consumer graphics boards
by Nvidia.
In our setup we used a stereo camera rig based on the microHDTV camera element provided by the Fraunhofer Institute
IIS. The cameras are very compact and support HD and Full HD resolutions at 24 to 60 Hz (synchronized). The stereo
camera rig is connected to a professional video capture board (DVS Centaurus II) via HD-SDI.
Figure 1. Reduced kiosk setup. Two players (left) are filmed using a stereoscopic camera-rig (middle).
The video streams are processed by a game engine using our multi-pass stereoscopic video
method. Only the left eye image is used for image recognition to track the players' movements.
The image on the screen shows the rendered scene as a composite of the mirrored players
augmented with virtual objects (right).
For rendering, we used a typical gaming PC running Microsoft Windows 7 Professional, with a Nvidia GeForce GTX
470 graphics card, the Intel Core i5 Quad-core processor, and eight gigabyte of memory. Our implementation of MSVR
was incorporated into Havok’s Vision Engine, a professional game engine that supports video textures and custom
render methods [19]. The engine provides a C++ API for development and supports Direct3D-rendering, which is
required to use the 3D Vision driver. It is widely used for commercial game titles, such as The Settlers 7 or Stronghold 3
[19].
Our system further allows splitting the camera image to display several parts of the image on different canvas objects.
Each split can be positioned completely independent within the scene. One application could be to identify the faces of
multiple players within a video and distribute the according parts throughout the game scene. It would also be possible to
dynamically change audience configurations to create new teams or new spatial conflicts. This system along with our
setup of stereoscopic parameters and calibration of the different parallaxes of the 3D video and the game engine scene is
described in [18].
Our multi-threaded implementation further supports motion recognition through simple subtraction of subsequent
images and through comparison with reference images. Despite the huge amount of data due to a S3D high definition
video stream of about 2.4 Gbit/s and the multi-pass rendering approach, this setup allows for interactive frame-rates of
25-30 fps.
5.2 Scenario
The game was intended to demonstrate applicability to sponsored gaming in a commercial movie theater event. Plus, our
test event took place during the Berlinale 2011 film festival. We thus decided for an appropriate movie theater scenario.
The background and the main game objects were designed with the according theme in mind. Still, the game concept's
objective to win a race does not fit into such a setting. We have never experienced a race in cinema. Hence, if this mix of
racing-based game concept and movie theater-based scenario results in a good game, probably many other themes
(brands, products, etc.) could be used. For the name of the game prototype, “YouDash3D” was chosen since it easily
identifies its major features: It is a race in which the player participates and it is presented in stereoscopic 3D.
Figure 1. YouDash3D: A stereoscopic 3D game that offers depth-based game mechanics and is easily
applicable to 3D movie theater settings. Please use anaglyph glasses (red/cyan) for full 3d quality.
5.3 Final Gameplay
The core game concept is the same as described before: two contestants race each other. In order to get closer to the
finish line, they must destroy objects which fly towards their positions. The players can induce damage to the flying
objects by creating movements in so-called trigger zones which are projected from the objects onto the player video
frame. The closer these objects explode to the player's depth, the faster he/she gets to the goal. The player who reaches it
first, wins.
One of the most significant decisions was to invert the race, so that the competitors would not run into the screen, but
coming out of the screen. We put the finish line slightly out of screen and let the effigies start deep in the background of
the scene, running towards the finish line into the foreground. This view better fits the scenario of letting the audience be
the runner than our initial approach, where the runners were seen from behind. First tests showed that seeing oneself in
3D video has a very positive effect. Starting with a small video and getting closer to the screen depth makes the video
larger and shows more rewarding detail. Flying objects also need less time to reach the player's video in the foreground
than in the background, making it the more difficult the closer a player effigy gets to the goal. The only shortcoming is
that the direction of the race is backwards and may seem awkward to first-time players. To help with understanding the
racing metaphor, it would be helpful to show the race track layout from a different, more natural, perspective before the
start of a race.
According to our concept there are only two distinct procedures players can perform to interact with the game: to move
in order to hit an object or to consciously avoid movement to prevent hitting objects at false depths. The latter procedure
is expected to be used by advanced players only. We accordingly created two groups of flying objects for which we
chose movie-related designs: popcorn buckets and Hollywood stars.
To avoid idle times, popcorn buckets can be triggered through their trigger zones as soon as they appear. To reward
depth precision, it was defined that the closer the object was to the original trigger zone when exploding, the higher the
amount of points rewarded and the greater the distance with which the player's image moved forward. To inform players
how well they performed this task, the distance to the zone would be indicated in form of four categorized and colorcoded text messages: perfect hit, near hit, getting closer, and missed.
The Hollywood stars can trigger special powers. To activate the stars, they need to be triggered exactly at correct depth.
If the player induced movement to a star's trigger zone, it would shrink significantly until it vanished without effect.
Only if triggered at the specified depth, the star fills up a special ability bar, displayed above the player's video. Once the
bar is filled, the ability is activated. The amount and the color of the bar filling up depend on the color and the size of the
star. So the largest effect comes from stars which have not yet been damaged. Three special abilities were designed:
boost/turbo (green), setting the other player back (red) and flipping the other player's effigy upside-down (blue). The last
ability proved to be very popular with players of the game (see Figure 2).
Figure 2. YouDash3D features vertical stereoscopic mirror. The right player is switched upside-down,
caused by a special ability of the left player. Now, hitting the trigger zones is much more difficult.
Please use anaglyph glasses (red/cyan) for full 3d quality.
This system strongly accounts for public play, supporting different stages of involvement, which we found useful in
other public multi-player situations: ad-hoc play, committed play, and involved play [20]. The popcorn buckets allow
immediate ad-hoc action. It is not critical if they explode at wrong depths, but committed players can increase the effect
by triggering them at the correct depth. The stars can be used by involved players who consciously control their actions
for strategic gameplay. For ad-hoc players, the stars have either no effect (if destroyed too early) or seem to add random
events, such as flipping a player upside down. Still, anyone can play and win the game without understanding all these
concepts.
Offering good visual feedback can help to increase involvement. The amount of damage induced to an object is indicated
through the amount of sparks emitted, while the player creates movement within the object's trigger zone. The overall
damage status is shown by the color of the sparks, turning from white to yellow and then red, until the object explodes.
The explosion of an object causes small stars to fly around, scaled in number according to its effect to the player.
The game difficulty can be varied through different object paths, sizes (indicating increased power), or speed. The
duration of a round can be changed by adding automatic braking: the players slowly go back into the background when
no object is destroyed until a maximum distance. Rubber-band-like behavior can be added to help weak players quickly
come back forward to increase competition. The order and type of flying objects can be thoroughly controlled for more
fine-tuned balance.
One helpful effect in our setup was to use a non-linear scale for the forward movements of the effigies: Players in the
foreground need to trigger more objects for the same amount of progress than in the background. As noted before, it is
very rewarding to see one's video in the foreground. The last few inches to reach the goal are the most thrilling ones.
5.4 Testing YouDash3D with the Public
To estimate the quality of the game experience, the prototype was evaluated informally at the PRIME symposium taking
place during the Berlinale 2011 in Berlin, Germany. A demonstration booth was set up and every interested workshop
attendee was given the chance to play the game. During play, participants were observed and afterwards queried for
comments. The majority of the symposium attendees were experts in the field of S3D video production. None of the
participants complained about improper 3D, the effect was received very positively and often called fascinating. Most of
them did not have prior experience with video games; some had never played a digital game before. For the initially
desired movie theater audience the same assumption has to be made.
The majority of people understood the gameplay after a maximum of one race and most stayed to play at least another
round afterwards. Younger players often stayed for several rounds, usually trying to achieve an entry in the highscore
list. Some people even queued up to play the game after they had observed others playing it. We take this as strong
indications that the goal of providing a fun experience for casual players was fulfilled.
The main reason for difficulties of understanding the game were the barely visible trigger zone objects associated with
each flying object. This problem could easily be solved by performing a few tests to determine optimal color and opacity
values for the zone objects. Other players had problems inducing continuous movements in the trigger zones. Instead,
they reached out for the objects and tried to cautiously touch them with barely noticeable movements. Here our simple
and 2D-based image processing failed.
This observation is mainly attributed to the two-player setting combined with the short viewing range. Since players
were able to perceive their movements across a relatively large depth range, they intuitively expected that the interaction
would span the same depth range.
Figure 3. YouDash3D Demonstration at the Prime 3D Symposium as part of the Berlinale film festival
2011, Berlin, Germany.
5.5 Transfer to Movie Theater Scale
The major difference of the current game concept to the 3D movie theater setting is that in the latter setting players
would be placed at varying depths. The additional depth in the camera image would have to be incorporated into the
game. This means the trigger zones would have to be placed at different depths according to the depths within the video
image and not only at the depth of the stereo image within the game scene.
To infer the position of the trigger zones, the vertical image positions would determine the distance to the camera,
similar to Sieber's at approach [15]. The reference distances would have to be determined in a calibration step for each
demonstration facility in advance. As a solution, a stereoscopic reference image of an empty cinema could be taken for
automatic depth recognition.
To apply to dynamic audience configurations, the game would need procedures to determine where trigger zones could
be created, not requiring interaction on empty seats. This could be achieved by a background image comparison.
Obviously, this method is very simple but suffers from lighting changes, which often occur in a cinema, depending on
the projected content. This problem would have to be resolved by deploying more advanced methods of image
processing or using heuristics.
Besides automatically identifying which seat is occupied, it would also have to be defined when and how the game
would start races automatically. As of now, the race would be started by an operator. While possible within the movie
theater scenario, this seems tedious and unnecessary. A very simple solution is to define a delay that determines when a
new race would start after one had ended. A more complex solution would interpret audience reactions.
As for the hardware setup, a good position of the stereo camera rig must be chosen. To convey the virtual mirror concept
to the audience, the camera pair should ideally be placed in the center of the screen. The cameras could also be placed
underneath the movie screen, as in our reduced kiosk setup. Possibly, audience members in the back might be occluded
by those in the front or have other problems participating.
Due to the size of the screen, a positioning in the center of the screen would indeed be possible, since audience members
would probably hardly notice small-sized cameras in front of the screen. How the rig would actually be placed, how it
would be connected to the computer, and how much the projector's light source would interfere with the motion
detection, are problems that remain to be solved. Another option could be to use multiple S3D camera rigs to capture the
entire audience.
The major obstacle preventing the realization of the scenario is the lighting setup in movie theaters. Projection based
display systems require dark environments to achieve ideal results. Current S3D display technologies even further
reduce the perceived light. At the same time, recording an image with a video camera requires properly lit scenery.
However, alternatives were considered during development as well. For example, the image captured by the camera
could be filtered stylistically to hide the effect of suboptimal image recording. Yet another version could use completely
artificial versions of the audience generated from the input image. Early ideas did not include effigies of the audience at
all, but represented them in some other way.
In addition to the display technology, sound is another important factor in play. Sound either serves to create an
atmosphere through background effects or music, or it can be used to direct attention to specific events. We have
previously experimented with a public multi-player game on a large multi-touch table [20]: When a player is unable to
visually understand the source of a sound, as it is outside his or her current focus, it becomes unclear what exactly might
have caused the sound. The signal thus becomes irrelevant and will probably be overheard. Based on these findings, we
recommend using sound as a global indicator, signaling important events or creating a common atmosphere. Feedback
on individual actions should be primarily visual.
6. CONCLUSIONS AND FUTURE WORK
In our paper, we worked out the promising case of interactive events in 3D movie theaters. By presenting our study in
detail, we hope to help others with similar goals. Unfortunately, we could not solve all problems, first to mention the
lighting issues in movie theaters. Here, brighter projection technology and other display setups (e.g. large LCD panel
technology) are expected to provide for a solution.
Despite, we thoroughly analyzed possible requirements for creating fun and interactive stereoscopic experiences with a
public movie theater audience. As our main contribution, we proposed a general concept which we explored in detail
using a prototype implementation. YouDash3D offers projected trigger zones, clustering of the audience video, and
depth-based game mechanics. It thus allows for an easy adaptation to a stereoscopic movie theater scenario without the
need for complex or extensive changes. YouDash3D includes detailed characteristics that favor the desired movie theater
setup, such as the easy to learn mechanics, the identification of individual impact, the non-mandatory invitation to play,
and the potential to encourage group dynamics at different levels of involvement. Most important of all, the game works
as a fun experience and could be easily adapted to other themes, e.g. suitable for advertisement. For product finalization,
methods of image processing and visual feedback design should be optimized beyond prototype status.
Offering FullHD live stereoscopic video in a real-time rendered S3D game scene, YouDash3D demonstrates novel and
fascinating presentation effects of S3D vision in games. Most importantly, the game is one of the first attempts to
actually utilize stereoscopic presentation for game design through depth-based game mechanics and rewarding
stereoscopic live-video. We believe and personally experienced this to be true.
However, this hypothesis would have to be investigated by a thorough evaluation that compares the performance and
game experience of subjects playing YouDash3D with a stereoscopic presentation versus those playing in monoscopic
view. Furthermore, the current implementation with its two-player setup offers interesting opportunities with depthbased interaction. The FullHD stereoscopic video stream also allows for a detailed depiction of the players, offering a
high quality immersion, naturally supporting spatial interaction of the players in the third dimension, as observed.
Hence, a perfect fit would be the integration of depth cameras, e.g. Microsoft's Kinect platform, or a real-time depth
map-creating algorithm to allow such interactions.
Besides improving the depth-based game mechanic, the use of video offers even more sophisticated options of image
processing, one to mention is face recognition. Especially in the described scenario, facial recognition could help to
reward single individuals from an audience by automatically extracting and highlighting their faces. In combination with
emotion recognition, selecting happy faces among the winning side and angry faces for the losers could offer more
personalized entertainment. In our development, we experimented with current systems but those failed as they were
unable to detect faces wearing shutter glasses.
7. ACKNOWLEDGEMENTS
This work has been co-funded by the BMWi through the Prime3D project. Vision Engine educational license was kindly
provided by Trinigy. We would like to thank DVS for providing the capture card, Fraunhofer IIS for the cameras and
their kind support with the test setting in their movie theater room.
REFERENCES
[1]
M. Zachara and J. P. Zagal, “Challenges for Success in Stereo Gaming : A Virtual Boy Case Study,” in
Proceedings of the International Conference on Advances in Computer Enterntainment Technology, pp. 99–106,
ACM (2009).
[2]
P. Nash, “3D Gaming in the Past,” PSNation, 2010, <http://www.psnation.org/2010/09/23/3d-gaming-in-thepast/> (2 December 2011).
[3]
K. L. Kroeker, “Looking beyond stereoscopic 3D’s revival,” Communications of the ACM 53, 14 (2010)
[doi:10.1145/1787234.1787241].
[4]
R. J. E. Rajae-Joordens, “Measuring experiences in gaming and TV applications,” Probing Experience, 77–90,
Springer (2008).
[5]
J. J. LaViola Jr and T. Litwiller, “Evaluating the benefits of 3d stereo in modern video games,” in Proceedings of
the 2011 annual conference on Human factors in computing systems, pp. 2345–2354, ACM (2011).
[6]
N. Holliman, “Mapping perceived depth to regions of interest in stereoscopic images,” Stereoscopic Displays
and Virtual Reality Systems XI, 1, 117–128, Spie (2004) [doi:10.1117/12.525853].
[7]
G. S. Hubona, P. N. Wheeler, G. W. Shirah, and M. Brandt, “The relative contributions of stereo, lighting, and
background scenes in promoting 3D depth visualization,” ACM Transactions on Computer-Human Interaction 6,
214–242 (1999) [doi:10.1145/329693.329695].
[8]
J. Schild and M. Masuch, “Fundamentals of Stereoscopic 3D Game Design,” in Proc. of the ICEC 2011, pp.
155–160, Ifip International Federation For Information Processing (2011).
[9]
D. Maynes-Aminzade, R. Pausch, and S. Seitz, “Techniques for interactive audience participation,” in ACM
SIGGRAPH 2002 conference abstracts and applications, p. 257, ACM, New York, NY, USA (2002)
[doi:http://doi.acm.org/10.1145/1242073.1242270].
[10]
L. C. Carpenter, “Method and apparatus for audience participation by electronic imaging.,” U.S. Patent No.
5210604 (1992).
[11]
L. C. Carpenter, “Video imaging method and apparatus for audience participation.,” U.S. Patent No. 5365266
(1992).
[12]
SS+K, “Newsbreaker live - interactive cinema game (YouTube video),” 2007,
<http://www.youtube.com/watch?v=y6izXII54Qc> (10 June 2011).
[13]
Advertising Carlton Screen, “Volvo Game in Cinema (YouTube video),”
<http://www.youtube.com/watch?v=cSm5NCbYbCY> (10 June 2011).
[14]
IO Design, “Asteroid Storm cinema interactive game (YouTube video),” 2009,
<http://www.youtube.com/watch?v=YLBEkxbQh6w&feature=related> (8 June 2011).
[15]
J. Sieber, S. McCallum, and G. Wyvill, “BallBouncer: Interactive Games for Theater Audiences,” in 13rd
International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia,
29 November - 2 December 2005, Dunedin, New Zealand, pp. 29–31 (2008).
[16]
Nvidia, “Nvidia 3D Vision Automatic - Best Practices Guide BPG-05394-001_v01” (2010).
[17]
J. Schild, S. Seele, J. Fischer, and M. Masuch, “Multi-pass Rendering of Stereoscopic Video on Consumer
Graphics Cards,” in Symposium on Interactive 3D Graphics and Games 8, pp. 209–209, ACM, New York, NY,
USA (2011) [doi:http://doi.acm.org/10.1145/1944745.1944789].
[18]
J. Schild, S. Seele, and M. Masuch, “Integrating Stereoscopic Video in 3D Games,” in Proc. of the ICEC 2011,
pp. 124–135, Ifip International Federation For Information Processing (2011).
[19]
Havok, “Havok Vision Engine,” 2011, <http://www.havok.com/products/vision-engine> (2 December 2011).
[20]
J. Schild and M. Masuch, “Game Design for Ad-Hoc Multi-Touch Gameplay on Large Tabletop Displays,” in
Proceedings of The International Academic Conference on the Future of Game Design and Technology, pp. 90–
97, ACM, New York, NY, USA (2010) [doi:http://doi.acm.org/10.1145/1920778.1920791].