Advanced Topics in Machine Learning
A. LAZARIC (INRIA-Lille)
DEI, Politecnico di Milano
SequeL – INRIA Lille
April 2-15, 2012
Outline
Elements of Statistical Learning Theory
Online Learning (a.k.a. Sequence Prediction)
Multi-armed Bandit Problem
Administrative Stuff
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 2/15
Elements of Statistical Learning Theory
Outline
Elements of Statistical Learning Theory
Online Learning (a.k.a. Sequence Prediction)
Multi-armed Bandit Problem
Administrative Stuff
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 3/15
Elements of Statistical Learning Theory
Machine Learning
Handwriting recognition
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Face detection/recognition
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Page ranking
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Graph analysis
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Sentiment classification
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Recommendation systems
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Recommendation systems
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Bioinformatics
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Robotics
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Robotics
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Natural Language Processing
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Natural Language Processing
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Machine Learning
Entertainment
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 4/15
Elements of Statistical Learning Theory
Elements of Statistical Learning Theory
What: Theoretical analysis of the behavior of a learning algorithm.
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 5/15
Elements of Statistical Learning Theory
Elements of Statistical Learning Theory
What: Theoretical analysis of the behavior of a learning algorithm.
Why: Learning algorithms work on noisy data (i.e., noisy data)
and thus their behavior is non-deterministic (but predictable!).
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 5/15
Elements of Statistical Learning Theory
Elements of Statistical Learning Theory
What: Theoretical analysis of the behavior of a learning algorithm.
Why: Learning algorithms work on noisy data (i.e., noisy data)
and thus their behavior is non-deterministic (but predictable!).
How: Overview on
I
Concentration of measures inequalities
I
VC–Theory for binary classification
I
L1-Regularized least–squares regression
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 5/15
Online Learning (a.k.a. Sequence Prediction)
Outline
Elements of Statistical Learning Theory
Online Learning (a.k.a. Sequence Prediction)
Multi-armed Bandit Problem
Administrative Stuff
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 6/15
Online Learning (a.k.a. Sequence Prediction)
Online Prediction
Binary prediction
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 7/15
Online Learning (a.k.a. Sequence Prediction)
Online Prediction
Online data compression
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 7/15
Online Learning (a.k.a. Sequence Prediction)
Online Prediction
Market prediction
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 7/15
Online Learning (a.k.a. Sequence Prediction)
Online Prediction
Betting systems
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 7/15
Online Learning (a.k.a. Sequence Prediction)
Online Prediction
Weather forecast
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 7/15
Online Learning (a.k.a. Sequence Prediction)
Online Prediction
Elections prediction
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 7/15
Online Learning (a.k.a. Sequence Prediction)
Online Prediction
Traffic forecast
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 7/15
Online Learning (a.k.a. Sequence Prediction)
Online Learning (a.k.a. Sequence Prediction)
What: Prediction of any sequence of elements in an online fashion
(e.g., stock market, binary sequences, weather, ...).
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 8/15
Online Learning (a.k.a. Sequence Prediction)
Online Learning (a.k.a. Sequence Prediction)
What: Prediction of any sequence of elements in an online fashion
(e.g., stock market, binary sequences, weather, ...).
Why: Most of the time data are not available in batches but are
provided in a stream and their behavior might change over time.
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 8/15
Online Learning (a.k.a. Sequence Prediction)
Online Learning (a.k.a. Sequence Prediction)
What: Prediction of any sequence of elements in an online fashion
(e.g., stock market, binary sequences, weather, ...).
Why: Most of the time data are not available in batches but are
provided in a stream and their behavior might change over time.
How: Overview on
I
Learning with expert advice for continuous prediction
I
Learning with expert advice for discrete prediction
I
Efficient forecasters
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 8/15
Multi-armed Bandit Problem
Outline
Elements of Statistical Learning Theory
Online Learning (a.k.a. Sequence Prediction)
Multi-armed Bandit Problem
Administrative Stuff
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 9/15
Multi-armed Bandit Problem
Online Prediction
Online web advertising
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 10/15
Multi-armed Bandit Problem
Online Prediction
Adaptive routing
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 10/15
Multi-armed Bandit Problem
Online Prediction
Clinical trials
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 10/15
Multi-armed Bandit Problem
Online Prediction
Computer games
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 10/15
Multi-armed Bandit Problem
Online Prediction
Oil/gas mining
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 10/15
Multi-armed Bandit Problem
Online Prediction
Adaptive marketing
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 10/15
Multi-armed Bandit Problem
Multi-armed Bandit Problem
What: Study of learning algorithms that are responsible for the
decisions and data acquisition at the same time.
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 11/15
Multi-armed Bandit Problem
Multi-armed Bandit Problem
What: Study of learning algorithms that are responsible for the
decisions and data acquisition at the same time.
Why: The observed data often depend on the learning system
itself. Thus, it should trade-off between collection of good and
informative data.
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 11/15
Multi-armed Bandit Problem
Multi-armed Bandit Problem
What: Study of learning algorithms that are responsible for the
decisions and data acquisition at the same time.
Why: The observed data often depend on the learning system
itself. Thus, it should trade-off between collection of good and
informative data.
How: Overview on
I
The exploration–exploitation dilemma
I
Stochastic bandits (UCB)
I
Non–stochastic bandits (Exp3)
I
Extensions to large set of arms
I
Connections to game theory
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 11/15
Administrative Stuff
Outline
Elements of Statistical Learning Theory
Online Learning (a.k.a. Sequence Prediction)
Multi-armed Bandit Problem
Administrative Stuff
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 12/15
Administrative Stuff
Schedule
Topic
Statistical Learning Theory
Statistical Learning Theory
Online Learning
Online Learning
Multi–armed Bandits
Multi–armed Bandits
Day/Time
Apr. 2nd, 9:00-13:00
Apr. 3rd, 9:00-13:00
Apr. 4th, 9:00-13:00
Apr. 11th, 9:00-13:00
Apr. 12th, 9:00-13:00
Apr. 13th, 9:00-13:00
A. LAZARIC – Advanced Topics in Machine Learning
Location
Sala Seminari
Sala Seminari
Sala Seminari
Sala Seminari
Sala Seminari
Sala Seminari
April 2-15, 2012 - 13/15
Administrative Stuff
Assignements
I
Option 1: Report on 2-3 papers (better if you propose them)
I
Option 2: Presentation on 2-3 papers (on the last day)
I
Option 3: Short project (if some interesting research
questions come out)
A. LAZARIC – Advanced Topics in Machine Learning
April 2-15, 2012 - 14/15
Advanced Topics in Machine Learning
Alessandro Lazaric
[email protected]
sequel.lille.inria.fr