Ask’nSeek : a new game for object detection and labeling
Ask’nSeek : a new game for object detection and
labeling
Axel Carlier
Oge Marques
Vincent Charvillat
October 7th 2012
1/25
Ask’nSeek : a new game for object detection and labeling
1
Games With A Purpose
Why games ?
Luis Von Ahn’s work
The ESP game
PeekaBoom
2
Ask’nSeek
3
What do we learn from Ask’nSeek ?
4
Results
5
Conclusion
2/25
Ask’nSeek : a new game for object detection and labeling
Games With A Purpose
Outline
1
Games With A Purpose
Why games ?
Luis Von Ahn’s work
The ESP game
PeekaBoom
2
Ask’nSeek
3
What do we learn from Ask’nSeek ?
4
Results
5
Conclusion
3/25
Ask’nSeek : a new game for object detection and labeling
Games With A Purpose
Why games ?
Reasons for using games
It’s one way of doing Crowdsourcing, i.e. bring users in the loop and
include the information given by their interactions in CV algorithms.
Games are non intrusive: users play only if they like the game.
Popular games gather thousands of players every day, which if exploited
correctly can give a potentially tremendous amount of data.
4/25
Ask’nSeek : a new game for object detection and labeling
Games With A Purpose
Luis Von Ahn’s work
The ESP Game is a game that helps labeling images.
5/25
Ask’nSeek : a new game for object detection and labeling
Games With A Purpose
Luis Von Ahn’s work
PeekaBoom is a game for locating objects in images.
6/25
Ask’nSeek : a new game for object detection and labeling
Ask’nSeek
Outline
1
Games With A Purpose
Why games ?
Luis Von Ahn’s work
The ESP game
PeekaBoom
2
Ask’nSeek
3
What do we learn from Ask’nSeek ?
4
Results
5
Conclusion
7/25
Ask’nSeek : a new game for object detection and labeling
What do we learn from Ask’nSeek ?
Outline
1
Games With A Purpose
Why games ?
Luis Von Ahn’s work
The ESP game
PeekaBoom
2
Ask’nSeek
3
What do we learn from Ask’nSeek ?
4
Results
5
Conclusion
8/25
Ask’nSeek : a new game for object detection and labeling
What do we learn from Ask’nSeek ?
Information collected from game logs
Labels reveal the objects present on the image
Spatial relations and clicks reveal the location of those objects
9/25
Ask’nSeek : a new game for object detection and labeling
What do we learn from Ask’nSeek ?
10/25
Ask’nSeek : a new game for object detection and labeling
What do we learn from Ask’nSeek ?
11/25
Ask’nSeek : a new game for object detection and labeling
What do we learn from Ask’nSeek ?
12/25
Ask’nSeek : a new game for object detection and labeling
What do we learn from Ask’nSeek ?
13/25
Ask’nSeek : a new game for object detection and labeling
What do we learn from Ask’nSeek ?
14/25
Ask’nSeek : a new game for object detection and labeling
What do we learn from Ask’nSeek ?
15/25
Ask’nSeek : a new game for object detection and labeling
Results
Outline
1
Games With A Purpose
Why games ?
Luis Von Ahn’s work
The ESP game
PeekaBoom
2
Ask’nSeek
3
What do we learn from Ask’nSeek ?
4
Results
5
Conclusion
16/25
Ask’nSeek : a new game for object detection and labeling
Results
User Study
40 participants
10 images
148 games played
3.1 indications per game
17/25
Ask’nSeek : a new game for object detection and labeling
Results
After 17 games:
Other labels: floor, body, pot, nose
18/25
Ask’nSeek : a new game for object detection and labeling
Results
19/25
Ask’nSeek : a new game for object detection and labeling
Results
20/25
Ask’nSeek : a new game for object detection and labeling
Results
21/25
Ask’nSeek : a new game for object detection and labeling
Conclusion
Outline
1
Games With A Purpose
Why games ?
Luis Von Ahn’s work
The ESP game
PeekaBoom
2
Ask’nSeek
3
What do we learn from Ask’nSeek ?
4
Results
5
Conclusion
22/25
Ask’nSeek : a new game for object detection and labeling
Conclusion
Conclusion
Ask’n Seek : a game for labeling images and detecting objects on images
It does in one game what ESP and Peekaboom do in two games
It takes only a few games per images to produce relevant outputs
It avoids explicitly asking the user to map labels and regions
23/25
Ask’nSeek : a new game for object detection and labeling
Conclusion
Ongoing and future work
Put the game into production
Gather data from a high number of users
Get results on a complete dataset
Thank you
Contact: [email protected]
http://tinyurl.com/askandseek
24/25
Ask’nSeek : a new game for object detection and labeling
Conclusion
25/25