Learning Automata
• Learns the unknown nature of an environment
• Variable structure stochastic learning automaton is
a quintuple {j,a,b,A,G} where:
– j(n), state of automaton; j={j1,…,js}
–
–
–
–
a(n), output of automaton; a={a1,…,ar}
b(n), input to automaton; b={b1,…,bm}
A, is the learning algorithm;
G[.], is the output function; a(n)=G[j(n)]
n indicates the iteration number.
Learning Automaton Schematic
Action
Environment
Response
b(n)
a(n)
Automaton
<f,a,b,A,G>
Output <states, actions, input, learning algorithm, output function>
Input
Probability Vector
• pj(n), action probability; the probability that
automaton is in state j at iteration n.
• Reinforcement scheme
If a(n) = ai and for j <> i; (j=1 to r)
pj(n+1) = pj(n) - g [pj(n)] when b(n)= 0.
pj(n+1) = pj(n) + h [pj(n)] when b(n)= 1.
• In order to preserve probability measure,
S pj (n) = 1, for j = 1 to r.
contd ...
• If a(n) = ai
r
pi(n+1) = pi(n) + S
g(pj(n)) when b(n) = 0
j=1, j<>i
r
pi(n+1) = pi(n) - S
h(pj(n))
j=1, j<>i
• g(.) is the reward function
• h(.) is the penalty function
when b(n) = 1
Schematic of Proposed Automata Model for
Mapping/Scheduling
Machine
HC System Model
Eresp
bsi(n)
asi(n)
Automaton for task si
<asi,bsi,Asi>
Output
<machines, environment response, learning algorithm >
Input
Model Construction
• Every task si associated with an S-model automaton
(VSSA).
• VSSA represented as {asi, bsi,Asi}, since r = s
– asi is set of actions asi
=
m0, m1, …, m|M|-1
– bsi is input to the automaton, bsi 
[0, 1]
closer to 0 – action favorable to system;
closer to 1 – action unfavorable to system
– Asi is reinforcement scheme
• pij(n) - action probability vector
– probability of assigning task si to machine mj
contd …
• Automata model for Mapping/Scheduling
– S-model VSSA is used
– Each automaton is represented as a tuple
{asi,bsi,Asi}
– asi = m0, m1, …, m|M|-1
– bsi  [0, 1]
(closer to 0 - favorable, 1 - unfavorable)
– If ck(n) is better than ck(n-1)
Ekresp = 0 else Ekresp = 1
– Translating Ekresp to bsi (n) requires two steps
Translating Ekresp to bsi
m|S|-11
E1resp
m01
E2resp
m0k
Ekresp
m1k
m11
Task s0
Task s1
bs 0
bs 1
Task s|S|-1
bs|s|-1
m|S|-1k
contd …
• Step 1: Translate Ekresp to msik (n), where
– msik (n) - input to automaton si with respect to cost
metric ck
– achieved by the heuristics
• Step 2: Achieved be means of Lagrange's
multiplier
|C|
bsi (n) = S lk * msik (n), i=1 to |S|-1;
j=1
where lk is the weight of metric ck
|C|
S lk = 1, lk > 0
j=1
Low Communication Complexity, Machines = 5
60000
60000
50000
50000
Cost
40000
40000
30000
30000
20000
20000
10000
10000
0
of
0.2
0
0
N
Weight
Ta
50
0.4
r
0.6
um
be
0.8
sk
s
0
5.74E+04+
5.23E+04 to 5.74E+04
4.72E+04 to 5.23E+04
4.21E+04 to 4.72E+04
3.70E+04 to 4.21E+04
3.20E+04 to 3.70E+04
2.69E+04 to 3.20E+04
2.18E+04 to 2.69E+04
1.67E+04 to 2.18E+04
1.17E+04 to 1.67E+04
6.58E+03 to 1.17E+04
1.50E+03 to 6.58E+03
Medium Communication Complexity, Machines = 5
70000
70000
60000
60000
50000
40000
40000
30000
30000
20000
20000
10000
10000
0
Weight
0.2
fT
50
0.4
0
0
ro
0.6
be
0.8
as
ks
0
Nu
m
Cost
50000
6.24E+04+
5.69E+04 to 6.24E+04
5.15E+04 to 5.69E+04
4.60E+04 to 5.15E+04
4.06E+04 to 4.60E+04
3.51E+04 to 4.06E+04
2.97E+04 to 3.51E+04
2.42E+04 to 2.97E+04
1.88E+04 to 2.42E+04
1.33E+04 to 1.88E+04
7.85E+03 to 1.33E+04
2.40E+03 to 7.85E+03
Schematic of Proposed Automata Model for
Architecture Trades
Machine
System Solution
Peval
bsi(n)
asi(n)
Automaton for component si
<asi,bsi,Asi>
Output <components, performance evaluation, learning algorithm >
Input
Model Construction
• Every component of the HW system si associated with
a P-model automaton (VSSA).
• VSSA represented as {asi, bsi,Asi}, since r = s
– asi is set of component types asi
=
c0, c1, …, c|M|-1
– bsi is input to the automaton, bsi = 0, 1
0 – performance favorable to system; 1 – unfavorable to system
– Asi is reinforcement scheme
• pij(n) - action probability vector
– probability of choosing component si from component type
cj
contd …
• Automata model for Architecture Trades
– P-model VSSA is used
– Each automaton is represented as a tuple
{asi,bsi,Asi}
– asi = c0, c1, …, c|M|-1
– bsi  0, 1
(0 - favorable, 1 - unfavorable)
– If ck(n) is better than ck(n-1)
Peval = 0 else Peval = 1
Conclusions
• Adaptive Framework for Mapping and
Architecture trades
• Automata models allow optimization of multiple
criteria
• Efficient / gracefully degradable solutions
• Framework construction suitable for tool
integration
– Mapping algorithm integrated with SAGETM
• Provides a basis for systems design from
application to the embedded HW