The model
Controllabilities
Minimal Intervention
Controllability Metrics in Markov Decision Linear
Models of Gene Networks
Dan Goreac1
Journées ALEA, CIRM, March 21st 2017
1 UPEM.
Based on papers joint with T. Diallo, M. Martinez (UPEM), E. -P.
Rotenstein (UAIC, Iasi, Romania), C. A. Grosu (UAIC, Iasi, Romania)
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Outline
1
The model
Biochemical Reactions
Mathematical Model
The Questions
2
Controllabilities
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
3
Minimal Intervention
Optimization Problems
Back to Lambda
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Lambda Phage
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Reference Model
Biochemical reactions
2R1
K1
R2 , D (+R2 )
DR2 (+R2 )
K4
K2
DR2 , D (+R2 )
K
K3
DR2 ,
K
DR2 R2 , DR2 + P !t DR2 + P + rR1 , R1 !d .
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Reference Model
Biochemical reactions
2R1
K1
R2 , D (+R2 )
K2
DR2 , D (+R2 )
K4
K
K3
DR2 ,
K
DR2 R2 , DR2 + P !t DR2 + P + rR1 , R1 !d .
DR2 (+R2 )
Groups of reaction
trend : DNA mechanism of the host E-Coli (D, DR2 , DR2 , DR2 R2 )T
K1
K
update : 2R1
R2 , R1 !d
K
rare : DR2 + P !t DR2 + P + rR1
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Reference Model
Biochemical reactions
2R1
K1
R2 , D (+R2 )
K2
DR2 , D (+R2 )
K4
K3
DR2 ,
K
K
DR2 R2 , DR2 + P !t DR2 + P + rR1 , R1 !d .
DR2 (+R2 )
Groups of reaction
trend : DNA mechanism of the host E-Coli (D, DR2 , DR2 , DR2 R2 )T
K1
K
update : 2R1
R2 , R1 !d
K
rare : DR2 + P !t DR2 + P + rR1
Some mathematical models (d1 , d2 , d3 , d4 , x1 , x2 ) :
pure jump, 2-scale PDMP, Marked-point, discrete model
Reaction speeds can be "chosen".
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Trend
At time n, trend (DNA occupation) is Ln
k
k
E.g. If Ln = D : D !2 DR2 , D !3 DR2
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Trend
At time n, trend (DNA occupation) is Ln
k
k
E.g. If Ln = D : D !2 DR2 , D !3 DR2
If Ln = D then Ln +1 =
(
k2
k 2 +k 3
k3
k 2 +k 3
DR2
DR2
Dan Goreac
with proportional probability
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Trend
At time n, trend (DNA occupation) is Ln
k
k
E.g. If Ln = D : D !2 DR2 , D !3 DR2
If Ln = D then Ln +1 =
(
k2
k 2 +k 3
k3
k 2 +k 3
DR2
DR2
with proportional probability
In general, since only one type of occupation, one gets basis vectors
e1 (D ), e2 (DR2 ) ... ep
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Trend
At time n, trend (DNA occupation) is Ln
k
k
E.g. If Ln = D : D !2 DR2 , D !3 DR2
If Ln = D then Ln +1 =
(
k2
k 2 +k 3
k3
k 2 +k 3
DR2
DR2
with proportional probability
In general, since only one type of occupation, one gets basis vectors
e1 (D ), e2 (DR2 ) ... ep
Take ∆Mn +1 = Ln +1 "average" (in fact E [Ln +1 /Fn ])
To make it simple, assume Ln +1 is completely independent of Ln
and has 0 mean
∆Mn +1 = Ln +1
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Update
2R1
K1
K
R2 , R1 !d
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Update
K1
K
R2 , R1 !d
continuous with choice of speed (u) :
x10 = k1 (u ) x12 kd (u ) x1 + k 1 (u ) x2
x20 = k1 (u ) x12 k 1 (u ) x2
2R1
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Update
K1
K
R2 , R1 !d
continuous with choice of speed (u) :
x10 = k1 (u ) x12 kd (u ) x1 + k 1 (u ) x2
x20 = k1 (u ) x12 k 1 (u ) x2
2R1
linearized
x10 = 2k1eq x1eq x1 kdeq x1 + k eq1 x2 + b 1 u
x20 = 2k1eq x1eq x1 k eq1 x2 + b 2 u
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Update
K1
K
R2 , R1 !d
continuous with choice of speed (u) :
x10 = k1 (u ) x12 kd (u ) x1 + k 1 (u ) x2
x20 = k1 (u ) x12 k 1 (u ) x2
2R1
linearized
x10 = 2k1eq x1eq x1 kdeq x1 + k eq1 x2 + b 1 u
x20 = 2k1eq x1eq x1 k eq1 x2 + b 2 u
or, again dXsx ,u = [A (γs ) Xsx ,u + Bs us ] ds
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Update
K1
K
R2 , R1 !d
continuous with choice of speed (u) :
x10 = k1 (u ) x12 kd (u ) x1 + k 1 (u ) x2
x20 = k1 (u ) x12 k 1 (u ) x2
2R1
linearized
x10 = 2k1eq x1eq x1 kdeq x1 + k eq1 x2 + b 1 u
x20 = 2k1eq x1eq x1 k eq1 x2 + b 2 u
or, again dXsx ,u = [A (γs ) Xsx ,u + Bs us ] ds
discrete Xnx+,u1 = An (ω ) Xnx ,u + Bun +1
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Rare (and Synthesis)
"
D
+R2
K2
DR2 ,
#
K
DR2 + P !t DR2 + P + rR1
Dan Goreac
.
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Rare (and Synthesis)
"
D
+R2
discrete
x1,n +1
x2,n +1
K2
=
DR2 ,
x1,n
x2,n
#
K
DR2 + P !t DR2 + P + rR1
+
Dan Goreac
0
0
r
0
.
x1,n
x2,n
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Rare (and Synthesis)
"
D
+R2
discrete
x1,n +1
x2,n +1
K2
=
DR2 ,
x1,n
x2,n
#
K
DR2 + P !t DR2 + P + rR1
+
0
0
r
0
.
x1,n
x2,n
Continuous f (con…g. DNA γ, reaction
R speeds u, fast variable X )
dXsx ,u = [A (γs ) Xsx ,u + Bs us ] ds + E C (γs , θ ) Xsx ,u q
e (d θds )
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Rare (and Synthesis)
"
D
+R2
discrete
x1,n +1
x2,n +1
K2
=
DR2 ,
x1,n
x2,n
#
K
DR2 + P !t DR2 + P + rR1
+
0
0
r
0
.
x1,n
x2,n
Continuous f (con…g. DNA γ, reaction
R speeds u, fast variable X )
dXsx ,u = [A (γs ) Xsx ,u + Bs us ] ds + E C (γs , θ ) Xsx ,u q
e (d θds )
Discrete
Xnx+,u1 = An (ω ) Xnx ,u + Bun +1 + ∑pi=1 h∆Mn +1 , ei i Ci ,n (ω ) Xnx ,u
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
Rare (and Synthesis)
"
D
+R2
discrete
x1,n +1
x2,n +1
K2
=
DR2 ,
x1,n
x2,n
#
K
DR2 + P !t DR2 + P + rR1
+
0
0
r
0
.
x1,n
x2,n
Continuous f (con…g. DNA γ, reaction
R speeds u, fast variable X )
dXsx ,u = [A (γs ) Xsx ,u + Bs us ] ds + E C (γs , θ ) Xsx ,u q
e (d θds )
Discrete
Xnx+,u1 = An (ω ) Xnx ,u + Bun +1 + ∑pi=1 h∆Mn +1 , ei i Ci ,n (ω ) Xnx ,u
The "expected" behavior is similar ... (is it ?)
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Biochemical Reactions
Mathematical Model
The Questions
The Questions
For the reference model, one has bistability
E.g. lytic : for some choice of speeds, lysis occurs (say at time T or
N ) i.e. XT = 0 (or, more general XT =target).
from arbitrary x to "almost" 0
from arbitrary x exactly to 0
from arbitrary x exactly/almost to any "possible" target
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Outline
1
The model
Biochemical Reactions
Mathematical Model
The Questions
2
Controllabilities
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
3
Minimal Intervention
Optimization Problems
Back to Lambda
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
A Hint to the Metric
In deterministic update
Xnx+,u1 = AXnx ,u + Bun +1 .
"the dual" YN =y , YnN ,y := AT YnN+,y1
Ei
D
E N 1 hD
un +1 , B T YnN+,y1 .
hXN , YN i = x, Y0N ,y + ∑
n =0
N ,ξ
In order for XNx ,u = 0, whenever all B T Yn +1 = 0, should have
Y0N ,y = 0
If "reversible" dynamics, Yn +1 = AT
B T AT
k
y0 = 0 for all k
controllability to 0 i¤ metric
k
T AT
y 7 ! y T ∑N
k =1 A BB
1
Yn
N should imply y0 = 0.
2
k
y
(in continuous case, application to power electronic actuator
placement Summers, Cortesi, Lygeros ’14)
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Towards Riccati-like Formulation
Recall that
Xnx+,u1 = An (ω ) Xnx ,u + Bun +1 + ∑pi=1 h∆Mn +1 , ei i Ci ,n (ω ) Xnx ,u
k
T AT
Let us look at ∑N
k =1 A BB
Recursively computed as PN = BB T ,
Pn = A 1 pn +1 + BB T
AT
k
1
How to deal with non-homogeneity (dependence on n) ?
1
Well ... Pn = An 1 Pn +1 + BB T
AT
n
How to deal with with stochasticity ?
Problem : too much information in pn +1 which is not available at
time n
Solution : decompose Pn +1 in what is known at time n and some
random variation
Couple (p, q ) .
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Riccati-like Formulation
Set pn +1 = "average"
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Riccati-like Formulation
Set pn +1 = "average"
Pn +1 pn +1 qn
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Riccati-like Formulation
Set pn +1 = "average"
Pn +1 pn +1 qn
Pn = An 1 pn +1 + BB T
h
AT
n
Dan Goreac
i 1
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Riccati-like Formulation
Set pn +1 = "average"
Pn +1 pn +1 qn
h i 1
Pn = An 1 pn +1 + BB T AT
n
Is it over ? Well ... NO :
Pn has some noise, and so does the process ) some correction
(covariance) term
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Riccati-like Formulation
Set pn +1 = "average"
Pn +1 pn +1 qn
h i 1
Pn = An 1 pn +1 + BB T AT
n
Is it over ? Well ... NO :
Pn has some noise, and so does the process ) some correction
(covariance) term
One "corrects" by substracting "horrible terms"
h i 1
αn := qn bruit 2
AT
n
ηn := qn bruit 3 + bruit 2
1
αT
n ηn α n
pn + 1
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Riccati-like Formulation
Set pn +1 = "average"
Pn +1 pn +1 qn
h i 1
Pn = An 1 pn +1 + BB T AT
n
Is it over ? Well ... NO :
Pn has some noise, and so does the process ) some correction
(covariance) term
One "corrects" by substracting "horrible terms"
h i 1
αn := qn bruit 2
AT
n
ηn := qn bruit 3 + bruit 2
1
αT
n ηn α n
pn + 1
Problem ηn is only semi-positive de…nite.
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Riccati-like Formulation
Set pn +1 = "average"
Pn +1 pn +1 qn
h i 1
Pn = An 1 pn +1 + BB T AT
n
Is it over ? Well ... NO :
Pn has some noise, and so does the process ) some correction
(covariance) term
One "corrects" by substracting "horrible terms"
h i 1
αn := qn bruit 2
AT
n
ηn := qn bruit 3 + bruit 2
1
αT
n ηn α n
pn + 1
Problem ηn is only semi-positive de…nite.
Solution : penalize by adding εI
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Horrible Terms (you may look away for 2 minutes)
In fact, Pnε +1 = E Pnε +1 /Fn + Qnε diag (∆Mn +1 ),
{z
}
{z
} |
|
p
Pnε = An 1 E Pnε +1 /Fn + BB T
αjn,ε :=
Qnε E
q
h
i 1
AT
n
1
αT
n,ε ηn,ε αn,ε ,
h i 1
∆Mn +1 , ej diag (∆Mn +1 ) /Fn AT
n
j ,k
ηn,
ε :=
εδj ,k Im m + 12 Qnε E h∆Mn +1 , ek i ∆Mn +1 , ej diag (∆Mn +1 ) /Fn
i
h
+ 12 E h∆Mn +1 , ek i ∆Mn +1 , ej (diag (∆Mn +1 ))T /Fn (Qnε )T
+E h∆Mn +1 , ek i ∆Mn +1 , ej /Fn E Pnε +1 /Fn
If C is present, even more "horrible terms" α, η.
JUST A,B condition does not su¢ ce R
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
Theorem
i. System is controllable to 0 , almost to 0 , lim inf P0ε
ε !0 +
0 (positive
de…nite).
ii. The norm is ky0 k2ctrl = lim inf P0ε y0 , y0 .
ε !0
Existence results :
iii. If An , Cn are non-random, 9P ε 0 (explicit).
iv. If Cn = 0, 9P ε 0 (explicit).
v. continuous and discrete conditions are NOT the same.
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Optimization Problems
Back to Lambda
Outline
1
The model
Biochemical Reactions
Mathematical Model
The Questions
2
Controllabilities
Metric By Observability
Riccati Formulation of the Metric
Main Theoretical Results
3
Minimal Intervention
Optimization Problems
Back to Lambda
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Optimization Problems
Back to Lambda
Scenarios, E¢ ciency
Several (r ) scenarios (Bi )i 2f1,..,r g : k k
kB kspec
ctrl : =
ctrl ,
B
ky k
inf y 6=0 kyctrlk ,B
ε
kB krank
ctrl : = Rank lim inf P0 (B )
ε !0
rank
controllable using B , kB kspec
ctrl > 0 , kB kctrl = m.
De…nitions
1)I is minimal spectral-e¢ cient intervention :
(i) kB (I)kspec
ctrl > 0 ;
(ii) 8J f1, ...r g, jJ j < jIj , one has kB (J )kspec
ctrl = 0;
(iii) 8J f1, ...r g, jJ j = jIj , one has kB (J )kspec
kB (I)kspec
ctrl
ctrl .
2) I is minimal rank-e¢ cient intervention :
(i) kB (I)krank
ctrl = m;
(ii) 8J f1, ...r g, jJ j < jIj , one has kB (J )krank
ctrl < m.
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Optimization Problems
Back to Lambda
Optimization Problems
maxI
f1,...r g
jIj=k
kB (I)kctrl , 1
k
r,
rank-based set functions are submodular (Lovasz ’83) :
f (S1 \ S2 ) + f (S1 [ S2 ) f (S1 ) + f (S2 )
submodularity is "a combinatorial analogue of concavity"
(Nemhauser ’78)
problem is NP-hard BUT a greedy approach provides good results.
SO : use k krank
ctrl and greedy heuristic ) minimal k
then, use k kspec
ctrl for such k.
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Optimization Problems
Back to Lambda
Back to the Initial Model
Xnx+,u1 = An (ω ) Xnx ,u + Bun +1 + ∑pi=1 h∆Mn +1 , ei i Ci ,n (ω ) Xnx ,u
A=
1
4
1
4
1
2
3
4
, C2 =
0
0
r
0
0
1
, respectively b2 =
1
1
direct control on dimer fails to work
b1 =
one needs SIMULTANEOUS control on monomer/dimer
BUT altering ONE external factor su¢ ces.
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks
The model
Controllabilities
Minimal Intervention
Optimization Problems
Back to Lambda
Thank you for your patience !
Dan Goreac
Controllability Metrics in Markov Decision Linear Models of Gene Networks