Nature-inspired Smart Info Systems
Ronald L. Westra, Ralf L. M. Peeters,
Department of Mathematics
Maastricht University
Robust Identification of Piecewise Linear
Gene-Protein Interaction Networks
NISIS conference, Albufeira, October 4, 2005
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
1
Nature-inspired Smart Info Systems
Items in this Presentation
1. Background and problem formulation
2. Modeling and identification of gene/proteins interactions
3. The implications of stochastic fluctuations and deterministic chaos
5. Example 1: Application on fission yeast expression data
5. Example 2: Application on artificial reaction model
5. Example 3: Application on Tyson-Novak model for fission yeast
6. Conclusions
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
2
Nature-inspired Smart Info Systems
1. Problem formulation
Questions:
* Can we identify (= reconstruct) gene regulatory networks from
time series of observations of (partial) genome wide and protein
concentrations?
* What is the influence of intrinsic noise and deterministic chaos
on the identifiability of such networks?
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
3
Nature-inspired Smart Info Systems
Problems in modeling and identification
Relation between mathematical model and phys-chem-biol reality
Macroscopic complexity from simple microscopic interactions
Approximate modeling as partitioned in subsystems with local
dynamics
Modeling of subsystems as piecewise linear systems (PWL)
PWL-Identification algorithms: network reconstruction from
(partial) expression and RNA/protein data
Experimental conditions of poor data: lots of gene but little data
The role of stochasticity and chaos on the identifiability
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
4
Nature-inspired Smart Info Systems
2. Modeling the Interactions between
Genes and Proteins
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
5
Nature-inspired Smart Info Systems
2.1 Modeling the molecular dynamics and
reaction kinetics as Stochastic Differential Equations
Biochemical reactions and kinematic rate equations:
this is a microscopic reality:
(in)elastic collisions,
electrostatic forces, “binding”
this is a statistic average.
true only under some conditions
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
6
Nature-inspired Smart Info Systems
Conditions for modeling reactions as rate equations
The law of large numbers. In inhomogeneous mixtures or in slow
reactions as in gene-, RNA-, and protein-interactions this will not
(always) be true. Hence; the problem is stochastic.
The Maxwell velocity distribution should apply, otherwise details of
the velocity distribution will enter. This condition is not met for
macromolecules in a cytoplasm.
The distribution of the internal degrees of freedom of the
constituents, like rotational and vibrational energies, must have
the same ’temperature’ as the Maxwell velocity distribution,
otherwise it will influence the rate of the collisions that result in a
chemical reaction. This condition is not met by gene/RNA/protein
interactions.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
7
Nature-inspired Smart Info Systems
Gene-Protein Interaction Networks as
PIECEWISE Linear Models
The general case is complex and approximate
Strongly dependent on unknown microscopic details
Relevant parameters are unidentified and thus unknown
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
8
Nature-inspired Smart Info Systems
Modeling of PWL Systems as
subspace models
Global dynamics:
Σ5
Local attractors
(uniform, cycles, strange)
Σ4
Basins of Attraction
Σ1
Σ3
Σ2
Σ6
Each BoA is a
subsystem Σi
“checkpoints”
State space
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
9
Nature-inspired Smart Info Systems
Modeling of PWL Systems as subspace models
State vector moves
through state space
driven by local
dynamics (attractor,
repeller) and inputs
in each subsystem Σ1
the dynamics is
governed by the local
equilibria.
approximation of
subsystem as linear
statespace model:
State space
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
10
Nature-inspired Smart Info Systems
PWL Systems as flexible networks
G
1
G
3
G
1
G
2
P
3
P
2
P
4
P
3
G
3
P
5
P
1
Σ1
Σ2
G
4
G
6
For different biological processes the subsystem defines a different
network structures
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
11
Nature-inspired Smart Info Systems
3. Identification of Interactions
between Genes and Proteins
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
12
Nature-inspired Smart Info Systems
The identifiability of Piecewise Linear models
from Microarray Timeseries
Sequence of genomewide expression profiles
at consequent instants
become more realistic
with decreasing costs …
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
13
Nature-inspired Smart Info Systems
Problems concerning the identifiability
of Piecewise Linear models
1. Due to the huge costs and efforts involved in the experiments, only
a limited number of time points are available in the data. Together with
the high dimensionality of the system, this makes the problem
severely under-determined.
2. In the time series many genes exhibit strong correlation in their
time-evolution, which is not per se indicative for a strong coupling
between these genes but rather induced by the over-all dynamics of
the ensemble of genes. This can be avoided by persistently exciting
inputs.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
14
Nature-inspired Smart Info Systems
Problems concerning the identifiability
of Piecewise Linear models
3. Not all genes are observed in the experiment, and certainly most of
the RNAs and proteins are not considered. therefore, there are many
hidden states.
4. Effects of stochastic fluctuations on genes with low transcription
factors are severe and will obscure their true dependencies.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
15
Nature-inspired Smart Info Systems
Problems with stochastic modeling
Such are the problems relating to the identifiability of
piecewise linear systems:
Are conditions for modeling rate equations met?
High stochasticity and chaos
Are piecewise linear approximations a valid metaphor?
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
16
Nature-inspired Smart Info Systems
The identification of PIECEWISE linear networks
by L1-minimization
K linear time-invariant subsystems {Σ1, Σ2, .., ΣK}
Continuous/Discrete time
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
17
Nature-inspired Smart Info Systems
4.2 The identification of PIECEWISE linear networks
by L1-minimization
Weights wkj indicate membership of observation #k
to subsystem
Σj :
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
18
Nature-inspired Smart Info Systems
Rich and Poor data
poor data: not sufficient empirical data is
available to reliably estimate all system parameters,
i.e. the resulting identification problem is underdetermined.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
19
Nature-inspired Smart Info Systems
(un)known switching times,
regular sampling intervals,
rich / poor data,
Identification of PWL models with known switching times and
regular sampling intervals from rich data
Identification of PWL models with known switching times and
regular sampling intervals from poor data
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
20
Nature-inspired Smart Info Systems
1. unknown switching times,
regular sampling intervals,
poor data, known state derivatives
This is similar to simple linear case
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
21
Nature-inspired Smart Info Systems
This can thus be written as:
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
22
Nature-inspired Smart Info Systems
with:
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
23
Nature-inspired Smart Info Systems
with:
The approach is as follows:
(i) initialize A, B, and W,
(ii) perform the iteration:
1. Compute H1 and H2, using the simple linear system approach
2. Using fixed W, compute A and B,
3. Using fixed A and B, compute W
until: (iii) criterion E has converged sufficiently –
or a maximum number of iterations.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
24
Nature-inspired Smart Info Systems
Linear L1-criterion:
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
25
Nature-inspired Smart Info Systems
With linear L1-criterion E1 the problem can be
formulated as LP-problem:
LP1: compute H1,H2 from simple linear case
LP2: A and B, using E1-criterion and extra constraints
for W, H1,H2,
LP3: compute optimal weights W, using E1-criterion
with constraints for W, H1,H2, A and B
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
26
Nature-inspired Smart Info Systems
2. unknown switching times,
regular sampling intervals,
poor data, unknown state derivatives
Use same philosophy as mentioned before
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
27
Nature-inspired Smart Info Systems
Subspace dynamics and linear L1-criterion :
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
28
Nature-inspired Smart Info Systems
System parameters and empirical data :
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
29
Nature-inspired Smart Info Systems
Quadratic Programming problem QP :
Problem: not well-posed: i.e.: Jacobian becomes
zero and ill-conditioned near optimum
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
30
Nature-inspired Smart Info Systems
Therefore split in TWO Linear Programming problems:
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
31
Nature-inspired Smart Info Systems
In case of sparse interactions replace LP1 with:
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
32
Nature-inspired Smart Info Systems
Performance of the robust Identification approach
Artificially produced data reconstructed with this approach
Compare reconstructed and original data
Here some results …
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
33
Nature-inspired Smart Info Systems
a: CPU-time Tc as a function of the problem size N,
b: Number of errors as a function of the number of nonzero entries k,
M = 150, m = 5, N = 50000.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
34
Nature-inspired Smart Info Systems
a: Number of errors versus M,
b: Computation time versus M
N = 50000, k = 10, m = 0.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
35
Nature-inspired Smart Info Systems
a: Minimal number of measurements Mmin required to compute A
free of error versus the problem size N,
b: Number of errors as a function of the intrinsic noise level σA
N = 10000, k = 10, m = 5, M = 150, measuring noise B = 0.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
36
Nature-inspired Smart Info Systems
The influence of increasing intrinsic noise on the identifiability.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
37
Nature-inspired Smart Info Systems
4. The Implications of Stochastic
fluctuations and Deterministic Chaos
4.1 Stochastic fluctuations :
* some experimental and numerical results
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
38
Nature-inspired Smart Info Systems
Stochastic fluctuations
the evolution of the expression of two coupled genes.
The genes, with expressions x1 and x2, are coupled as:
with zero-mean Gaussian stochastic noise.
Influence of increasing intrinsic noise level.
The time steps dt relate to the strength of the noise.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
39
Nature-inspired Smart Info Systems
Influence of stochastic fluctuations on the evolution of the expression of two coupled genes.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
40
Nature-inspired Smart Info Systems
Noise-induced control in
single-cell gene expression
i. In experimental work on E. coli,
Elowitz and Swain found that low
intracellular copy numbers of molecules
can limit the precision of gene regulation.
They found that:
genotypic identical cells exhibit
substantial phenotypic variation
this variation arises from stochasticity
in gene expression
this variation is essential for many
biological processes
prime factors in stochasticity are:
transcription rate, regulatory dynamics,
and genetic factors .
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
41
Nature-inspired Smart Info Systems
ii. In stochastic simulations
on Drosophila and
Neurospora, Goldbeter and
Gonze found that robust
circadian oscillations can
emerge at the cellular level,
even when only a few tens of
mRNA and protein molecules
are involved. This shows how
autoregulation processes at
the cellular level allow the
emergence of a coherent
biological rhythm out of
molecular noise.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
42
Nature-inspired Smart Info Systems
iii. Steuer found that the
addition of noise to a
deterministic simulation model of
the cell-cycle in fission yeast
(Tyson-Novak model) could
explain several experimental
findings, such as the existence
of quantized cycle times in
double-mutant wee1−cdc25
cells.
Moreover, he found that his
stochastic model led to the
emergence of noise induced
oscillations.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
43
Nature-inspired Smart Info Systems
4. The Implications of Stochastic
fluctuations and Deterministic Chaos
4.1 Stochastic fluctuations :
* some experimental and numerical results
4.2 Deterministic chaos:
* some remarks
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
44
Nature-inspired Smart Info Systems
Gene-Protein system with coupling strength λ
Consider a gene with expression x(t) that is coupled to
an stimulating protein with density a(t).
x(t  1)  .a(t ).(1  x(t ))
 x(t )  xc 
a(t  1)  f 




With f a sigmoid function.
Now consider the limit gene states x(t) as function of λ
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
45
Nature-inspired Smart Info Systems
Feigenbaum bifurcation as λ increases
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
46
Nature-inspired Smart Info Systems
Complex deterministic chaotic behaviour
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
47
Nature-inspired Smart Info Systems
Deterministic Chaos in a Gene-Protein system
This chaotic behaviour is beneficial for
identification as it provides many independent
data points to explore the dynamics in the basin
of attraction.
In this way, chaos acts as the persistently
exciting inputs in the linear-convergent case.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
48
Nature-inspired Smart Info Systems
Applications
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
49
Nature-inspired Smart Info Systems
Example 1: how to apply this method on current data sets
Spellman et al. data for cell-cycle of fission yeast :
Components: 6179 genes measured for 18-24 irregular
time instants
Processing: fuzzy C-means, gene annotation with Go
term finder and Fatigo, net recontruction with identification
algorithm
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
50
Nature-inspired Smart Info Systems
Spellman et al. data for cell-cycle of fission yeast :
Processing:
Selection of most up/down-regulated genes: 3107 from 6179
Clustering: fuzzy C-means: best outcome 23 clusters
Gene annotation with Go term finder (4th level) and Fatigo,
both for biological process and cellular component
Net recontruction with identification algorithm on 23 clusters
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
51
Nature-inspired Smart Info Systems
Centroids after clustering 23 clusters
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
52
Nature-inspired Smart Info Systems
Gene ontology
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
53
Nature-inspired Smart Info Systems
Gene ontology
Cluster 1
GO Term Finder: The genes are involved in spindle pole during the cell cycle, with relations to microtubuli and
chromosomal structure.
FatiGO: The main cellular component is the chromosome.
Cluster 2
GO Term Finder: The genes are involved in proliferation and replications, especially bud neck and polarized
growth.
FatiGO: The results found by the GO Term Finder are confirmed.
…………….
Cluster 22
GO Term Finder: Only a few annotations are found and there are many unknown genes. The genes are
involved in respiration and reproduction. The main cellular components are the actin/cortical skeleton and the
mitochondrial inner membrane.
FatiGO: No further clear annotations are found.
Cluster 23
GO Term Finder: The genes are involved in RNA processing. The main cellular components are the nucleus,
the RNA polymerase complex and the ribonucleoprotein complex.
FatiGO: The main cellular component is the ribonucleoprotein complex.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
54
Nature-inspired Smart Info Systems
Reonstructed network
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
55
Nature-inspired Smart Info Systems
Example 2: artificial data of hierarchic/sparse network
Artificial reaction network with:
Components:
2 master genes with high transcription rates
3 slave genes with low transcription rates
4 agents (= RNA or proteins).
Processes: stimulation, inhibition, transcription, and
reactions between ‘agents’
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
56
Nature-inspired Smart Info Systems
Dynamics:
– large hierarchic and sparse network
– implicit relation between genes with expression x
through agents (= proteins, RNA) with concentration a
– system near equilibrium and small perturbations
– inputs: persistent excitation u
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
57
Nature-inspired Smart Info Systems
Dynamics:
– implicit system dynamics:
– linear statespace model makes gene interaction explicit:
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
58
Nature-inspired Smart Info Systems
Dynamics:
– estimate gene-gene interaction matrix A from
empirical data:
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
59
Nature-inspired Smart Info Systems
reactions
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
60
Nature-inspired Smart Info Systems
reactions
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
61
Nature-inspired Smart Info Systems
reactions
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
62
Nature-inspired Smart Info Systems
rate equations
Matlab-simulation
y(1)
y(2)
y(3)
y(4)
y(5)
y(6)
y(7)
y(8)
y(9)
=
=
=
=
=
=
=
=
=
-
0.03*x(1)
0.05*x(2)
0.02*x(3)
0.01*x(4)
0.02*x(5)
0.02*a(1)
0.01*a(2)
0.01*a(3)
0.05*a(4)
+
+
+
+
+
+
+
0.2*(1-x(1))*a(2)^2 - 0.2*x(1)*a(3) ;
0.3*(1-x(2))*a(1)
- 0.1*x(2)*a(4) ;
0.1*(1-x(3))*a(2)
- 0.1*x(3)*a(1) ;
0.2*(1-x(4))*a(1)*a(2) - 0.2*x(4)*a(3)^2;
0.3*(1-x(5))*a(3)
- 0.1*x(5)*a(1);
0.4*x(1) - 0.2*a(1)*a(2) - 0.1*a(1)*a(3)^3;
0.15*x(2) - 0.2*a(1)*a(2);
+ 0.2*a(1)*a(2) - 0.1*a(1)*a(3)^3;
+ 0.9*a(1)*a(3);
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
63
Nature-inspired Smart Info Systems
Real network structure: implicit
a
1
2
b
3
c
d
g
gene
p
agent
4
5
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
64
Nature-inspired Smart Info Systems
Real network structure: explicit
master
master
1
2
3
4
5
slave
slave
slave
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
65
Nature-inspired Smart Info Systems
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
66
Nature-inspired Smart Info Systems
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
67
Nature-inspired Smart Info Systems
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
68
Nature-inspired Smart Info Systems
Reconstructed network structure:
low noise
master
master
1
2
3
4
slave
slave
5
slave
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
69
Nature-inspired Smart Info Systems
Reconstructed network structure:
moderate noise
master
master
1
2
3
4
5
slave
slave
slave
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
70
Nature-inspired Smart Info Systems
Reconstructed network structure:
high noise (an example)
slave
master
1
3
slave
2
4
5
slave
master
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
71
Nature-inspired Smart Info Systems
Example 3: influence of noise
Artificial model for infection with deterministic chaos
and no intrinsic noise :
Components:
2 agents (= RNA or proteins).
Processes: stimulation, inhibition, transcription, and
reactions between ‘agents’
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
72
Nature-inspired Smart Info Systems
Example 3: data of Tyson-Novak math. model for cell cycle
Tyson-Novak model for cell-cycle of fission yeast :
Components:
9 agents (= RNA or proteins).
Processes: stimulation, inhibition, transcription, and
reactions between ‘agents’
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
73
Nature-inspired Smart Info Systems
The deterministic Tyson-Novak model.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
74
Nature-inspired Smart Info Systems
The stochastic Tyson-Novak model.
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
75
Nature-inspired Smart Info Systems
Example: stochastic Tyson-Novak model
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
76
Nature-inspired Smart Info Systems
Example: stochastic Tyson-Novak model
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
77
Nature-inspired Smart Info Systems
5. Conclusions
The piecewise linear system is an attractive metaphor
for modeling dynamic gene-protein interactions
Robust identification can efficiently reconstruct network
structures of PWL systems for ‘poor’ data
Stochastic fluctuations mostly affect slave genes with
low transcription rates
Strongest links (e.g. master genes) are most resistant to
increasing noise
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
78
Nature-inspired Smart Info Systems
Discussion …

G
1
G
3
G
1
G
2
P
3
P
2
P
4
P
3
G
3
P
5
P
1
Σ1
Σ2
G
4
Westra: Robust Identification of Piecewise Linear Gene-Protein Interaction Networks
G
6
79