Collective Intelligence:
The Power of Many
Collective Intelligence
Slide 1
Database Management, Spring 2007
The Wisdom of Crowds
„
Under some conditions, group IQ > individual IQ
Examples:
„
Guess the weight of a bull: 542.9 kg (543.4 kg)
„
Guess the number of sweets in a jar: 871 (850)
„
Find the way out of a maze:
z
First individual attempt avg 34.3 turns
z
Second individual attempt avg 12.8 turns
z
Majority turn decision avg 9 turns
z
Group decision is better than best individual.
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
Slide 2
Database Management, Spring 2007
The 1986 Challenger Crash
Built by 4 contractors:
Lift off: 11:38 EST
Crash: 11:39
Rockwell:
shuttle and engines
„
Lockheed:
ground support
„
Martin Marietta:
external fuel tank
„
Stock prices of the 4 companies in the following 6 hours
Morton Thiokol:
fuel booster rocket
„
R, L, MM
crash
MT
Collective Intelligence
Slide 3
Database Management, Spring 2007
Characteristics of Wise Crowds
„
„
Crowds are not wise when there is:
z
Peer pressure
z
Group think
z
Herd behavior
Crowds are wise when there is:
1.
Diversity of opinion
2.
Independence
3.
Decentralization
4.
Aggregation
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
Internet
dynamics
Entrepreneurial
challenge
Slide 4
Database Management, Spring 2007
Collective intelligence in action: atom shopping VS bit shopping
Amazon’s edge: collective buying, rating & reviews
Collective Intelligence
Slide 5
Database Management, Spring 2007
Collective intelligence in action: searching a directory VS searching the crowd wisdom
Google edge: pageRank, based
on collective linking preferences
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
Slide 6
Database Management, Spring 2007
Collective intelligence in action: buying from a stranger
eBay edge: collective reputation building
Collective Intelligence
Slide 7
Database Management, Spring 2007
Collective citing
(PageRank
motivation)
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
Slide 8
Database Management, Spring 2007
Collective citing (cont.)
Collective Intelligence
Slide 9
Database Management, Spring 2007
Collective tagging
Tag: a word you use to describe an object. Unlike folders, you make up tags
when you need them, and you can use as many as you like
„ Folders
impose a
hierarchy
„ Tags
span
a network
„ Taxonomy
VS
Folksonomy
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
Slide 10
Database Management, Spring 2007
Collective tagging (cont.)
Other examples of
collective contents
tagging:
Collective Intelligence
Slide 11
Database Management, Spring 2007
Collective intelligence: human computing
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
Slide 12
Database Management, Spring 2007
Collective intelligence: human computing (cont.)
„ Social
aspects
„ Human
computation.
Collective Intelligence
Slide 13
Database Management, Spring 2007
Collaborative filtering
First generation:
„
Learn what other people say about an object of interest
„
Find what is interesting from what other people say
Second generation:
„
Learn / Find what people like you say.
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
Slide 14
Database Management, Spring 2007
Blockbusters VS Sleepers
Blockbusters
„
Dominated by a few superstars
(authors, actors, singers)
„
Promoted nationally and aggressively
„
Consumption is supply-driven
„
Consumption is based on familiarity and lazy
search
Sleepers
„
Rise through word of mouth by opinion leaders
„
Opinion leaders don’t pick sleepers;
They create them.
Collective Intelligence
Slide 15
Database Management, Spring 2007
Where do you go for product recommendations?
„
„
„
Commodities:
z
Expert opinions: Consumer Reports, Car & Driver, Sound & Vision, …
z
User opinions: Zagat, J.D. Powers, …
Media products:
z
People you trust
z
People who know you
z
People who passed your taste test
Collaborative filtering: a system that sifts through the opinions and
preferences of numerous consumers and systematically finds (a) people
whose tastes are similar to yours and (b) what they like.
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
Slide 16
Database Management, Spring 2007
Collaborative filtering example: Movielens
Based on the following
“discovery”:
In order to know what
someone wants, you
need to know what
they've wanted.
Collective Intelligence
Slide 17
Database Management, Spring 2007
Collaborative filtering example: Movielens (cont.)
Observations:
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
„
Ease of rating
„
Ease of testing
„
Ease of Learning
„
Social aspects
Slide 18
Database Management, Spring 2007
CF can be used to:
„
Shape tastes: it offers options that you would never consider
otherwise
„
Create and hawk new products and services
„
Give gifts optimally (now, gifts = blockbusters, to avoid uncertainty)
„
Empower location-aware cell phones
„
Create social networks
„
Predict group tastes as well as individual tastes.
Collective Intelligence
Slide 19
Database Management, Spring 2007
CF: food for thought
„
Unlike rigid stereotyping and demographics, CF is dynamically changing
„
CF goes against our tendency to over-simplify, classify, and stereotype
„
“People are starved for real advice, desperate for a recommendation from
someone they know and who they feel knows them” (Gladwell)
„
CF = the future of marketing
„
CF and the Long Tail.
Further reading:
z
The Science of the Sleeper (course reading pack)
z
The Tipping Point (book)
z
The Wisdom of Crowds (book)
Collective Intelligence
Copyright 2007 © Shimon Schocken and Maya Elhalal
Slide 20
Database Management, Spring 2007