Machine Learning 참고 자료
Learning

Definition

Learning is the improvement of performance in
some environment through the acquisition of
knowledge resulting from experience in that
environment.
2
Machine Learning: Tasks

Supervised Learning

Learn fw from training set D={(x,y)} s.t.
f w (x)  y  f (x)

Classification: y is discrete
Regression: y is continuous


Unsupervised Learning



Learn fw from D={(x)} s.t.
Density Estimation f w (x)  x
Compression, Clustering
3
Machine Learning: Methods

Symbolic Learning


Neural Learning


Genetic Algorithms
Probabilistic Learning


Multilayer Perceptrons (MLPs)
Evolutionary Learning


Version Space Learning
Bayesian Networks (BNs)
Other Machine Learning Methods

Decision Trees (DTs)
4
Applications of Machine
Learning

Driving an autonomous vehicle


Classifying new astronomical structures


무인 자동차 운전, 센서기반 제어 등에도 응용
천체 물체 분류, Decision tree learning 기법 사용
Playing world-class Backgammon

실제 게임을 통해서 전략을 학습, 탐색공간 문제에
응용
5
A Definition of Learning
: Well-posed Learning Problems

Definition




A computer program is said to learn from experience E
with respect to some class of tasks T and performance
measure P, if its performance at tasks in T, as measured
by P, improves with experience E.
A class of tasks T
Experience E
Performance measure P
6
Checkers Problem (1/2)




말은 대각선으로만 움직일 수 있다.
맞은편 끝까지 가기 전에는 앞으로만 진행할 수 있다.
대각선에 상대편 말이 있을 경우 그 말을 없앨수 있다.
게임은 한편 말이 모두 없어지면 끝난다.
7
Checkers Problem (2/2)

homepage


http://www.geocities.com/Heartland/7134/Green/grprechecker.htm
http://www.acfcheckers.com
8
A Checkers Learning Problem

Three Features: 학습문제의 정의




The class of tasks
The measure of performance to be improved
The source of experience
Example



Task T: playing checkers
Performance measure P: percent of games won against
opponent
Training experience E: playing practice games against
itself
9
Designing a Learning System




Choosing the Training Experience
Choosing the Target Function
Choosing a Representation for the Target
Function
Choosing a Function Approximation
Algorithm
10
Choosing the Training
Experience (1/2)

Key Attributes

Direct/indirect feedback



Direct feedback: checkers state and correct move
Indirect feedback: move sequence and final
outcomes
Degree of controlling the sequence of training
example

Learner가 학습 정보를 얻을 때 teacher의 도움을
받는 정도
11
Choosing the Training
Experience (2/2)

Distribution of examples

시스템의 성능을 평가하는 테스트의 예제 분포
를 잘 반영해야 함
12
Choosing the Target Function
(1/2)

A function that chooses the best move M for
any B



ChooseMove : B  M
Difficult to learn
It is useful to reduce the problem of
improving performance P at task T to the
problem of learning some particular target
function.
13
Choosing the Target Function
(2/2)

An evaluation function that assigns a
numerical score to any B

V : B R
14
Target Function for the
Checkers Problem

Algorithm




If b is a final state that is won, then V(b) = 100
……. that is lost, then V(b)=-100
……. that is drawn, then V(b)=0
If b is not a final state, then V(b)=V(b’), where
b’ is the best final board state
15
Choosing a Representation for
the Target Function
^

Describing the function V





Tables
Rules
Polynomial functions
Neural nets
Trade-off in choice


Expressive power
Size of training data
16
Linear Combination as
Representation
^
V(b) = w0 + w1x1 + w2x2 + w3x3 +w4x4 + w5x5 + w6x6
x1: # of black pieces on the board
x2: # of red pieces on the board
x3: # of black kings on the board
x4: # of red kings on the board
x5: # of black pieces threatened by red
x6: # of red pieces threatened by black
w1 - w6: weights
17
Partial Design of a Checkers
Learning Program





Task T: playing checkers
Performance measure P: Percent of games won in
the world tournament
Training experience E: games played against itself
Target function V: Board  R
Target function representation
^

V(b) = w0 + w1x1 + w2x2 + w3x3 + w4x4 + w5x5 + w6x6
18
Choosing a Function
Approximation Algorithm

A training example is represented as an ordered
pair <b, Vtrain(b)>



b: board state
Vtrain(b): training value for b
Instance: “black has won the game (x2 = 0)
<<x1=3, x2=0, x3=1, x4=0, x5=0, x6=0>, +100>
19
Choosing a Function
Approximation Algorithm

Estimating training values for intermediate
board states
^
 Vtrain(b)  V (Successor(b))
^

V : current approximation to V

Successor(b): the next board state
20
Adjusting the Weights (1/2)


Choosing wi to best fit the training examples
Minimize the squared error
E
2
(
V
train
(
b
)

V
'
(
b
))

b ,Vtra in( b ) trainingexample
21
Adjusting the Weights (2/2)

LMS Weight Update Rule
For each training example <b, Vtrain(b)>
1. Use the current weights to calculate V’(b)
2. For each weight wi, update it as
^
wi  wi   (Vtrain (b)  V (b)) xi
22
Sequence of Design Choices
Determine Type of
Training Experience
Games
against experts
Games against
self
Table of
correct moves
Determine
Target Function
Board
 move
Polynomial
Board
 value
Determine Representation
Of Learned Function
Linear function
of six features
Arfiticial NN
Determine
Learning Algorithm
Gradient
descent
Complete Design
Linear
Programming
23
Perspectives in ML


“Learning as search in a space of possible
hypotheses”
Representations for hypotheses




Linear functions
Logical descriptions
Decision trees
Neural networks
24
Perspectives in ML

Learning methods are characterized by their
search strategies and by the underlying
structure of the search spaces.
25
Summary

기계학습은 다양한 응용분야에서 실용적 가치
가 크다.



많은 데이터로부터 규칙성을 발견하는 문제(data
mining)
문제의 성격 규명이 어려워 효과적인 알고리즘을
개발할 지식이 없는 문제 영역(human face
recognition)
변화하는 환경에 동적으로 적응하여야 하는 문제
영역(manufacturing process control)
26
Summary

기계학습은 다양한 다른 학문 분야와 밀접히
관련된다.


인공지능, 확률통계, 정보이론, 계산이론, 심리학,
신경과학, 제어이론, 철학
잘 정의된 학습 문제는 다음을 요구한다.

문제(task)의 명확한 기술, 성능평가 기준, 훈련경험
을 위한 사례
27
Summary

기계학습 시스템의 설계 시에는 다음 사항을
고려 하여야 한다.




훈련경험의 유형 선택
학습할 목표함수
목표함수에 대한 표현
훈련 예로부터 목표함수를 학습하기 위한 알고리
즘
28
Summary

학습은 가능한 가설 공간에서 주어진 훈련 예
와 다른 배경지식을 가장 잘 반영하는 하나의
가설을 탐색하는 탐색이다.

다양한 학습 방법은 서로 다른 가설공간의 형태와
이 공간 내에서 탐색을 수행하는 전략에 의해 규정
지어진다.
29
Neural Networks
Biological motivation



Neuron receives signals from other neurons through its
dendrites
Transmits signals generated by its cell body along the
axon
Network of Neuron
31
Neural Network Representations

The primitive unit(e.g. perceptron)


A learning process in the ANN


N input signals  weighted sum  threshold function
 generate an output
Learning process involves choosing values for the
weights w0, …, wn
Learning rules

How network weights are updated?
32
Gradient descent and the delta
rule

The delta rule

Linear unit for which the output o is given by

o( x)  w  x
Measure for the training error of a hypothesis




d : the set of traing examples
td : the target output for training example d
od : the output of the linear unit for training example d
We can characterize E as a function of w
33
Gradient descent and the delta
rule
34
Gradient descent and the delta
rule

Derivation of the gradient descent rule

Direction of steepest descent along the error
space

Derivative E with respect to each component of w

The negative of this vector therefore gives the
direction of steepest decrease  E(w)
35
Gradient descent and the delta
rule

Training rule for gradient descent

wi ← wi + wi

Efficient way of calculating the gradient

where,
So, wi    (td  od ) xid
d D
36
Gradient descent and the delta
rule


If  is too large, the gradient
descent search runs the risk
of overstepping the
minimum
gradually reduce the value
of 
37
Multilayer Networks

Why multilayer network?


Single perceptrons can only express linear decision
surfaces
So, add an extra(hidden) layer between the inputs and
outputs

E.g.) the speech recognition task
38
Multilayer Networks

Sigmoid function
39
Error Function for BP
 

1
E ( w) 
(t kd  okd ) 2
2 dD koutputs

E defined as a sum of the squared errors
over all the output units k for all the
training examples d.
40
BP Algorithm
41
Learning Until…



After a fixed number of iterations (epochs)
Once the error falls below some threshold
Once the validation error meets some
criterion
42
Self Organizing Map
Introduction


Unsupervised Learning
SOM (Self Organizing Map)


Visualization
Abstraction
44
SOM structures
Output Layer
Neighborhood
Input Layer
45
Data to be clustered
46
After 100 iterations
47
After 500 iterations
48
After 2000 iterations
49
After 10000 iterations
50