DSCE launch event, December 2, 2013
Increasing user satisfaction
by diversification of
recommended item sets
Dr. Ir. Martijn C. Willemsen
TU/e HTI Group
[email protected]
Mark P. Graus, MSc.
Adversitement B.V - AUDACIS & TU/e HTI Group
[email protected]
Online study on user satisfaction and difficulty
Choice Overload
People are more attracted to larger items sets, but find it harder to
choose from such sets and show lower choice satisfaction (Iyengar
and Lepper, 2000)
Matrix Factorization recommender based on MovieLens dataset: all
movies from 1994 (5.6M ratings for 70k users and 5.4k movies).
Movies shown with title and predicted rating.
Design: 2x3 between subjects design: 2 list sizes: 5 and 20 and
3 levels of diversification: none (top 5/20), medium, high
Less attractive
30% sales
Higher purchase
satisfaction
More attractive
3% sales
Attractiveness and Difficulty
Choice overload is the result of an
interplay between attractiveness
(larger set provides more benefits
of choice) and choice difficulty (larger
set has more opportunity costs:
more comparisons needed, more
potential regret).
Procedure: 159 Participants from an online database
• Rating task to train the system (15 ratings)
• Choose one item from a list of recommendations
• Report subjective perception (Perc. diversity / attractiveness)
and experience (choice difficulty /satisfaction)
Results: Structural equation model
Diversity manipulation increases perceived diversity, reduces
difficulty and increases attractiveness. Choice satisfaction is indeed
an interplay between attractiveness (pos) and diffculty (neg).
Using diversification to reduce choice overload
A recent meta-analysis (Scheibehenne et al., 2010) shows choice
overload is not omnipresent, but stronger when there are no strong
prior preferences and little differences in attractiveness between
items. Prior psychological studies did not control for the diversity of
the item set.
Latent Feature Diversification
Predicted rating:
rank in top-200
Map users and items to
latent factor space
• item is a vector qi
• user a vector pu
Satisfaction and choices from the list
With higher diversity, people pick items with lower ranks, thus
resulting in less ‘optimal’ choice in terms of predicted rating but
without a reduction in choice satisfaction! Satisfaction increases with
diversification for 5 item list and remains similar for 20 items.
rˆui  q pu
T
i
100
80
60
40
20
0
5 items
20 items
none
Implementation:
10-dim. MF model
Take personalized top-200
Diversification
Select K items with
highest inter-item distance
(using city-block)
Satisfaction
Rank of chosen option
med
high
2-dimensional latent factor space with diversification
avg. rating
Rating of chosen option
4.8
4.6
4.4
4.2
4
3.8
none
med
diversity
high
standardized score
RQ: Can we reduce choice difficulty and choice overload by
using personally diversified sets, controlling for attractiveness?
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
5 items
20 items
none
med
diversity
high
Conclusions
Medium: select most diverse
from 100 items closest to top
Diversity reduces choice difficulty and can improve choice
satisfaction, even when the diversified list has movies with lower
predicted ratings than standard top-N lists.
High: select most diverse from
all items in top-200
Offering personalized diversified small items sets might be the
key to help decision makers cope with too much choice!
/ Human-Technology Interaction, IE&IS