Aisha Hasan
Essay 3
Pranav Mistry’s Sixth Sense and John Underkoffler’s G-Speak are about making digital
interactions more physical so we can get rid of the “digital divide”, making computation more
natural even if technology becomes more complex.
Underkoffler works with manipulating things in 3D using projectors and special gloves for
motion sensing in his “spatial operating environment” where several people can collaborate. He
demonstrates this by browsing, searching and sorting photos in 3D, snipping people out of movies
and ‘dragging’ them onto another plane, where they reanimate.
Mistry takes this one step further: taking digital representations and making them physical
instead of taking physical things and digitizing them, thus eliminating the need to learn to interact
with machines.
He started off by head mounted projector and gesture tracking cameras, and now has a
system where any surface can be used to compute: walls, floors, paper and even our palms. We can
draw, change colour and zoom in and out using our hands, intuitively. The camera understands what
we are holding in our hand (e.g. it will read a book’s title, match it to known books online, and
search for reviews). Bits of information could be dragged on to paper from various sources,
manipulating it and later transferring it to a full size computer with a ‘pinch’ gesture.
Interaction with everyday objects goes further: a digital projection of a ‘map' will show
location of an object placed on it (e.g placing a cup or boarding pass highlights restaurant or
boarding gate locations respectively). Newspapers will have projections of live videos and real time
weather updates, boarding passes will show updated time delays, ping-pong played on a train floor,
photos can be taken and time checked through gestures.
He intends to bring the technology to the masses rather than confine it to a lab by providing
open source software (as the hardware is inexpensive).
Link: http://www.qatar.cmu.edu/~aishah/Essay3.pdf