The distribution mimic
problem
Qiaoyi Fang
2016/08/13
Problem:
Given discrete distributions X
and Y, mimic Y with X.
Define a “Die”.
Here is distribution X (P(1), P(2), ..., P(n)).
Let's carved a die with n faces numbered 1, 2, ..., n.
th
Face i is top with probabiliy P(i) (in accordance
with distribution X).
We have rolled K times die X and reached the destination
layer.
route sign on
route u->j
Goal: Let P_dest approxiamte to Distribution
NEW
Goal:
Let
E
minimize!
Y!
di , j the expected(optimal) probability of ith
layer, jth node
ei , j the error made on ith layer, jth node
E
how big “error” P_dest have made
：From discrete to continuous!
super-sigmoid function
sigmoid function
How to get derivative of
minimize?
in oder to
Back Propagation & Chain Rule
Not yet!
Take happens,
repeat route
intovery
account!
When
repeat
let Esigns
be very
huge!
So finally, the gradient is
Let
Brouhaha&Cantata Network
Acknowledgements
Q&A
• Thanks for listening !