Logo
the feasibility of machine
learning
Logo

Component of learning
Formalization
–
Input（输入）:X (customer application)
think of it as deed dimension vector
–
Output（输出）:Y(+1,-1)
good/bad customer
–
Target Function(目标函数) : f :x→y
ideal credit approval formula
Company Logo
Logo

Component of learning
Formalization
–
–
Data（数据）: (𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ),…, (𝑥𝑛 , 𝑦𝑛 )
historical records
↓
↓
↓
Hypothesis(假设) ：g :x→y
为了得到目标函数的公式
F is unknown G is very much known
actually we created it
Company Logo
Logo
Component of learning
UNKNOWN TARGET FUNCTION
f :x→y
↓ ↓
TRAINING EXAMPLES
(𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ),…, (𝑥𝑛 , 𝑦𝑛 )
FINAL HYPOTHESIS
g
(G hopefully approximates F)
Company Logo
Logo
Component of learning
UNKNOWN TARGET FUNCTION
f :x→y
↓ ↓
TRAINING EXAMPLES
(𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ),…, (𝑥𝑛 , 𝑦𝑛 )
FINAL HYPOTHESIS
g
(G hopefully approximates F)
Company Logo
Logo
Component of learning
TRAINING EXAMPLES
(𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ),…, (𝑥𝑛 , 𝑦𝑛 )
LEARNING ALGORITHM →HYPOTHESIS SET
(从现实模型公式中创造公式） （将它们成为假设集）
FINAL HYPOTHESIS
Company Logo
Logo
Component of learning
HYPOTHESIS SET H
从假设集选出一个假设
H 衍生出一堆H’s(待定函数)
Company Logo
Logo
Component of learning
UNKNOWN TARGET FUNCTION
f :x→y
↓ ↓
TRAINING EXAMPLES
(𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ),…, (𝑥𝑛 , 𝑦𝑛 )
FINAL HYPOTHESIS
g
(G hopefully approximates F)
Company Logo
Logo

HYPOTHESIS SET
Hypothesis Set
–

H = {h}
g∈ H
Learning Algorithm
–
Together. they are referred to as the
learning model.
a hypothesis set and a learning algorithm
Company Logo
Logo
A simple hypothesis set—’perceptron’

For input X= (𝑥1 , … 𝑥𝑑 ) attributes of a
customer
–
Approve credit if
𝑑
𝑖=1 𝑤𝑖 𝑥𝑖
–
> threshold
Deny credit if
𝑑
𝑖=1 𝑤𝑖 𝑥𝑖
< threshold
This linear formula h∈ H can be written as
h(x) = sign( (
𝑑
𝑖=1 𝑤𝑖 𝑥𝑖 )
– threshold )
Company Logo
Logo
Learning Feasible
Company Logo
Logo

Learning Feasible
A related experiment
P(picking red)=μ
P(picking green)=1-μ
μ=probability of
red marbles
Company Logo
Logo
Learning Feasible
Pick N marbles independently
The fraction of red marbles in sample = v
Company Logo
Logo

Does v Say anything about μ?
NO!
Sample can be mostly green while bin
is mostly red
Company Logo
Logo

Does v Say anything about μ?
Yes
Sample frequency v is close to bin
frequency μ
P v − 𝜇 > ε ≤ 2𝑒
−2𝜀2 𝑁
This is called Hoeffding Inequality
Company Logo
Logo

Learning Feasible
Bin
–

Unknown is a number μ
Learning
–
Unknown is a function f:x→y
each marble is a point x ∈ X
Company Logo
Logo

Learning Feasible
Bin
–

Unknown is a number μ
Learning
–
Unknown is a function f:x→y
each marble is a point x ∈ X
Hypothesis got it right
h(x)=f(x)
Company Logo
Logo

Learning Feasible
Bin
–

Unknown is a number μ
Learning
–
Unknown is a function f:x→y
each marble is a point x ∈ X
Hypothesis got it wrong
h(x)≠f(x)
Company Logo
Logo
Learning Feasible
UNKNOWN TARGET FUNCTION
f :x→y
↓ ↓
TRAINING EXAMPLES
(𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ),…, (𝑥𝑛 , 𝑦𝑛 )
FINAL HYPOTHESIS
g
(G hopefully approximates F)
Company Logo
Logo
Learning Feasible
Probability distribution
P on X
Company Logo