Extracting
Valuable
Information
Lazily
Shiry Ginosar
One picture is worth a
thousand words
If you have never seen
an image, a description
might help your
imagination
Imagine a picture.
It is a painting of two lovers.
A man and a woman.
Only their faces and necks are
shown but they are covered
with white sheets.
In the background there is
greenery and blue skies with
white clouds.
The lovers
Rene Magritte, 1928
Most of the web is not
accessible to the
visually impaired
Why?
1. Images
captions
at all.
2. There
required
do not have proper
or are not captioned
are images that are
for navigation.
Our task is to attach a
description to every
image on the web
Wait…
Didn’t
already do that!?
Not really…
only provides
word labels.
These are great for image search
but not enough for
accessibility
Labels:
Eat, Drink, Man, Woman
Labels: Man, Woman
Our task is to attach a
descriptive
paragraph
description
to everyto
every
image
on the
image
on the
webweb
Following the EVIL
tradition, this will be
done in the form of a
FUN GAME
Picture
Treasure Hunt
Input Output behavior
Descriptive
paragraph
image
Game
The game is an internet
race between multiple
players all aiming to
find the same randomly
chosen image.
They are the “seekers”
The only information
they have about this
image comes from a
written description that
is given to them
We provide them with an
image search engine
Because searching is cool
A typical seeker screen
drawing man woman
The image is a
drawing of a
man and a
woman…
One of the players is
the “describer”. He is
the only one who gets to
see the image.
He will provide us with the
description we are looking for
A typical describer
screen
The image is a
drawing of a
man and a
woman…
Once a seeker finds the
correct image, he
becomes the next
describer and a new
image is chosen
Points
The seeker
that finds
the image
first
The
describer, in
proportion to
the time it
took the
seekers to
find the
image
We are testing other
ideas to make this more
fun
• Taboo words
• Intentional corruption of
search results
• A mega puzzle for each game
This form of a FUN GAME
will hopefully provide
us with descriptions of
all images on the web
Thus contributing to the
accessibility of the web
One picture is worth a
thousand words
Shiry Ginosar
Advisor: Professor Manuel Blum
Graduate Mentor: Luis von Ahn
Brought to you by EVIL
Labels: Man, Woman
Labels:
Dali, portrait, photograph