Distributed Control
The Importance of Signals and
Boundaries[边界]
Nothing is less real than realism. Details are confusing. It is only by selection,
by elimination, by emphases that we get at the real meaning of things.
--Georgia O’Keefe
Outline
1)
Introduction to Distributed Control.
(pp. 3-14)
2)
Reaction Networks.
(pp. 15-25)
3)
Urn[缸] models of uncontrolled reactions.
(pp. 26-31)
4)
Hierarchical[层级] control.
(pp. 32-42)
5)
Markov models of hierarchical control. (pp. 43-48)
Introduction to Distributed Control (1)
In traditional control theory:
An external controller directs a system (“plant”) using signals
generated by the plant and external feedback.
Distributed Control occurs when the control originates internally through
interaction of components of the system
Networks of interaction (Reaction Networks) have a central role in the
study of Distributed Control.
Introduction to Distributed Control (2)
Adaptive Control contrasted to Distributed Control
Adaptive
Control
Fitness = 1/operating cost
Distributed
Control
Different agents have different controls.
Connections between agents are frequently
made and broken.
Introduction to Distributed Control (3)
Adaptive control in many complex systems, such as flight controllers, is often
implemented by a program -- a linked set of IF/THEN rules.
As we will see, reaction nets can also be represented by a linked IF/THEN rules,
and they are distributed.
Reaction nets are a natural way to study distributed control.
Introduction to Distributed Control (4)
Reactions in biological cells are primarily controlled through enzymes[酶]
(catalysts[催化剂]) and a hierarchy of enclosures by semi-permeable membranes
(selective filters).
[半-渗透膜]
This hierarchy yields adaptive control orders of magnitude better than
we can obtain with artificial systems.
The details of the hierarchy are immensely complex.
Even comparative physiology[比较生理学], comparing very simple cells
to much more complex organisms, gives only vague ideas about the hierarchy.
Introduction to Distributed Control (5)
Spontaneous[自发的] Emergence of Levels
Spore
孢子
differentiation
of cells
[细胞分化]
water
single cell
no water
aggregation
of cells
Slime mold (Discoideum) [粘液菌]
When the environment is unfavorable, individual cells
aggregate, form boundaries[边界], and specialize.
Introduction to Distributed Control (6)
Genetically-specified signal/boundary interactions in plants:
 mitochondria[线粒体](energy conversion),
chloroplasts (photosynthesis[光合作用])
mycorrhizal fungi[真菌] (nutrient and water uptake)
debris-decomposing bacteria (recycling)
pollinators[授粉者] (e.g., bees),
seed-transporting organisms (e.g., fruit eaters)
predators [捕食者] (e.g., plant eaters)
Signals and boundaries control the network of interactions at each level of
the hierarchy.
It is difficult to analyze these interactions using traditional mathematics.
Introduction to Distributed Control (6)
Genetically-specified signal/boundary interactions in plants:

mitochondria[线粒体](energy conversion),
chloroplasts (photosynthesis[光合作用])
mycorrhizal fungi[真菌] (nutrient and water uptake)
debris-decomposing bacteria (recycling)
pollinators[授粉者] (e.g., bees),
seed-transporting organisms (e.g., fruit eaters)
predators [捕食者] (e.g., plant eaters)
Signals and boundaries control the network of interactions at each level of
the hierarchy.
It is difficult to analyze these interactions using traditional mathematics.
Introduction to Distributed Control (6)
Genetically-specified signal/boundary interactions in plants:

mitochondria[线粒体](energy conversion)
chloroplasts (photosynthesis[光合作用])
mycorrhizal fungi[真菌] (nutrient and water uptake)
debris-decomposing bacteria (recycling)
pollinators[授粉者] (e.g., bees),
seed-transporting organisms (e.g., fruit eaters)
predators [捕食者] (e.g., plant eaters)
Signals and boundaries control the network of interactions at each level of
the hierarchy.
It is difficult to analyze these interactions using traditional mathematics.
Introduction to Distributed Control (6)
Genetically-specified signal/boundary interactions in plants:

mitochondria[线粒体](energy conversion)
chloroplasts (photosynthesis[光合作用])
mycorrhizal fungi[真菌] (nutrient and water uptake)
debris-decomposing bacteria (recycling)
pollinators[授粉者] (e.g., bees),
seed-transporting organisms (e.g., fruit eaters)
predators [捕食者] (e.g., plant eaters)
Signals and boundaries control the network of interactions at each
level of the hierarchy.
It is difficult to analyze these interactions using traditional mathematics.
Introduction to Distributed Control (6)
Genetically-specified signal/boundary interactions in plants:

mitochondria[线粒体](energy conversion)
chloroplasts (photosynthesis[光合作用])
mycorrhizal fungi[真菌] (nutrient and water uptake)
debris-decomposing bacteria (recycling)
pollinators[授粉者] (e.g., bees)
seed-transporting organisms (e.g., fruit eaters)
predators [捕食者] (e.g., plant eaters)
Signals and boundaries control the network of interactions at each level of
the hierarchy.
It is difficult to analyze these interactions using traditional mathematics.
Introduction to Distributed Control (6)
Genetically-specified signal/boundary interactions in plants:

mitochondria[线粒体](energy conversion)
chloroplasts (photosynthesis[光合作用])
mycorrhizal fungi[真菌] (nutrient and water uptake)
debris-decomposing bacteria (recycling)
pollinators[授粉者] (e.g., bees),
seed-transporting organisms (e.g., fruit eaters)
predators [捕食者] (e.g., plant eaters)
Signals and boundaries control the network of interactions at each level of
the hierarchy.
It is difficult to analyze these interactions using traditional mathematics.
Introduction to Distributed Control (6)
Genetically-specified signal/boundary interactions in plants:

mitochondria[线粒体](energy conversion)
chloroplasts (photosynthesis[光合作用])
mycorrhizal fungi[真菌] (nutrient and water uptake)
debris-decomposing bacteria (recycling)
pollinators[授粉者] (e.g., bees),
seed-transporting organisms (e.g., fruit eaters)
predators [捕食者] (e.g., plant eaters)
Signals and boundaries control the network of interactions at each level of
the hierarchy.
It is difficult to analyze these interactions using traditional mathematics.
Introduction to Distributed Control (7)
Even beyond biology, it is important to understand hierarchical
distributed control because it occurs in many different complex
adaptive systems (cas).
Systems where signal/boundary interactions are critical:








Biological cell (chromosome/protein communication; organelles)
Ecosystem
Geopolitics [地缘政治]
Reaction network (with phases and/or membranes[膜])
Central Nervous System regions
Language
Markets (tags)
Psychology (induction and discovery)
Introduction to Distributed Control (8)
Spontaneous Boundary Formation
Before
After
厌水蛋白质不能溶解在水里，
但是在油里是可溶解的并且粘
在一起
A boundary is the interface between two
distinct sets of reactants.
(E.g., the boundary between Mandarin[普
通话] and Cantonese[广东话].)
Later, boundaries will be represented by tagged urns.
Introduction to Distributed Control (9)
As is usual in theoretical physics, we try to use simplified exploratory models
to obtain ideas about where to look in real systems.
Then we formulate critical experiments to test our hypotheses.
As we will see, tagged urn models [带标志的缸模型 ] are helpful in studying
distributed control.
Introduction to Distributed Control (10)
Tagged urn models of hierarchical control offer the following
advantages:
1) Complicated hierarchies of enclosures and distributed control are
easily presented.
2) There is a relevant mathematics, Markov processes, and a relevant
search technique, the Monte Carlo algorithm.
3) Using Monte Carlo algorithms, it is easy to find communities of
reactants[反应物] (niches[小生境]) -- communities that persist
because of recirculation[再流通] and autocatalysis[自催化].
4) Because all reactants and membranes are presented as strings, the
adaptive co-evolution of tagged urn models is easily simulated
using a genetic algorithm.
Introduction to Distributed Control (11)
In a complex adaptive system (cas) , where there are multiple interacting
agents that learn , control is necessarily distributed.
Boundaries separate agents into communities (modularity),
allowing specialization and higher efficiency.
Signals allow one community to partially control others.
Signals and Boundaries are both modified by evolution.
Introduction to Distributed Control (12)
Distributed Control of Reaction Nets
All cas can be presented as networks of reactions (rule based agents) and
resource flows.
Reaction networks give us a single formalism for studying both cas
and distributed control.
After reviewing reaction networks we will see how boundaries and
signals make possible distributed control of networks.
Overview of Reaction Networks (1)
Reaction networks result from spatially distributed sets of interactions
between molecules, signals, and/or resources.
E.g., a collection of reactions of the kind A+B F C+D distributed
over a 2-dimensional geometry.
Overview of Reaction Networks (2)
The reactants [反应物] (e.g. molecules) have tags [标志] (active sites) that
determine the reactions that take place.
Reactants come into contact through random elastic collisions.
“Billiard-ball” [撞球] mechanics.
Reactants in contact react with a probability determined by the reaction
rate.
An urn model [缸模型] of the state of the reaction network
allows a Markov matrix analysis.
Overview of Reaction Networks (3)
Objectives
1)
Observe the robustness and organization of the evolving reaction
networks.
2)
Observe the kinds of agents (coordinated sets of reactions), if
any, that form through the evolution of tags.
3)
Develop a concept of niche [小生境] (local resource enrichment)
suitable for dynamic, perpetually [永久地] changing networks.
These objectives have a major role in understanding any cas.
Overview of Reaction Networks (4)
Emergent Phenomena Expected
1) Boundaries arise through constraints provided by tags on reactants.
Boundaries allow the formation of agents with individual histories.
2) Agents become building blocks for still more complex agents.
A ‘layered’ use of tags evolves (similar to the membrane, organelle, cell,
organ, … hierarchy of biological cells).
3) Cycles in the reaction net provide locally increased concentrations [浓度] of
reactants.
The resulting niches [小生境] offer possibilities for agent exploitation.
4) Increasing diversity of the rules, signals, and agents arises as the reaction
network evolves (under a genetic algorithm).
A Quick Review of Reaction Networks
(1)
Reactants
Reactants are classified according to active sites (tags).
Reactants diffuse and collide at random (a ‘billiard ball’ mechanics), undergoing mass
action chemistry depending upon their active sites.
A Quick Review of Reaction Networks
(2)
A Binary Reaction
R1 + R2 F R3 + R4
A Quick Review of Reaction Networks
(3)
A Binary Reaction Based on Tags
[连接酶]
[受体]
prefix
[前缀]
A Quick Review of Reaction Networks
(4)
A Sequence of Binary Reactions
A Quick Review of Reaction Networks
(5)
The Effect of Recycling (Material Feedback)
A Quick Review of Reaction Networks
(9)
Summarizing: Reactions, Conditions, & Tags
A reaction requires a given combination of tags (active sites) for each
input reactant; any reactant with that combination can serve as that
input.
The combination of tags required for a reaction can be specified by using
using ‘don’t care’ (#) symbols for the non-tag locations (as in a
classifier rule).
A Quick Review of Reaction Networks
(7)
Embedding a Simple Ecosystem in a Reaction Network
Substrate [培养基]: {a, b, c, #}
multiple copies of each element
Plant: a + a + a + b + b + c => aababc;
aa# => aa#
Herbivore [食草动物]: aa# + b+ b => bb#c + a + a;
bb# => bb#
Carnivore [食肉动物]: bb# +c + c => cc# + b + b;
cc# => cc#
Bacterium [细菌]: #c => {elements in string #};
#c => #c
[The two reactions in the bacterium compete]
Uncontrolled Reactions
In uncontrolled reactions, the reactants are uniformly mixed and reactions
occur when the reactants undergo elastic collisions.
Ui = reactant type i, i = 1, …, M.
ni(t) = number of copies of reactant type i at time t.
N = Si n i = total number of copies of reactants, a constant.
pi(t) = ni(t)/N = the proportion of reactant type i at time t.
rij = the proportion of collisions resulting in a reaction between reactants
i and j = the forward reaction rate.
An Urn Model for Uncontrolled Reactions (1)
In this urn model, the number of balls in urn Uj is proportional to pj .
Assume that the reaction between reactants indexed by Uh and Ui
produces the products Uj and Uk.
The probability that the products will be produced is given by the reaction
equation
pj = pk = rhiphpi
An Urn Model for Uncontrolled Reactions (2)
The state of the system at time t is S(t) = ( n1(t), n2(t), …, nM(t)),
= the number of balls in each urn
In the urn model, to go from S(t) to S(t+1), pick two balls at random from
the urns and produce the products with probability rhi.
If the products are produced
S(t+1) = ( …, nh(t)-1, …, ni(t)-1, …, nj(t)+1, … nk(t)+1, … ),
else
S(t+1) = S(t).
An Urn Model for Uncontrolled Reactions (3)
This simple urn model can be presented as a Markov process with one
dimension for each possible distribution of balls in the urns.
Notation: Let [X,Y] be the number of different ways of choosing
Y objects from a total of X objects.
There are b = [N+M-1, M-1] distinct distributions of N balls in M urns.
b is easily derived by considering the number of binary
numbers of length N+M-1 having exactly M-1 ones,
(For example, with N=3 and M=2,
0100
=> 1 ball in the first urn
2 balls in the second urn.)
An Urn Model for Uncontrolled Reactions (4)
R is a bxb matrix having Ruv as its component at row h and column i.
Row u corresponds to one possible distribution of balls in urns,
and column v represents a possible product distribution
Let S(t), the current state, correspond to row u of the matrix.
Let S(t+1) = ( …, nh(t)-1, …, ni(t)-1, …, nj(t)+1, … nk(t)+1, … ),
a possible result of a random draw from the distribution S(t),
correspond to column v.
Then Ruv = rhiphpi.
Note that the distribution corresponding to row u fully
specifies phand pi .
Note that several different draws can yield the result S(t+1).
An Urn Model for Uncontrolled Reactions (5)
As is usual with a Markov representation
S(t+T) = S(t)RT
and the equilibrium distribution of reactants is given by the eigenvector S*
corresponding to the positive eigenvalue e of the matrix R
S*(t+1) = eS*(t).
A Spatial Urn Model
Each site (square in the array) contains a set of urns representing the reactant
types present in that area.
The number of balls in each urn gives the local concentration of that
reactant type.
Each reactant species (same active sites) is assigned a distinct color.
Diffusion takes place by moving balls at random between the urns.
Urn Models of Tag-based Reactions
Control via
Semi-permeable Membranes
A semi-permeable membrane is a filter. It allows only reactants with specific tags
to diffuse from one side of the membrane to the other.
Outside
Inside
Outside: High diversity of reactants with low individual concentrations
=> Low reaction rates
Semi-permeable boundary filters flow
=> Increased reaction rates for selected reactants
=> Locally increased concentration of selected reactants
Inside: Increased concentrations
=> Catalysts become effective as "switches" determining
reaction sequences (much like a computer program)
Urn Model of a Semi-permeable Membrane
Each urn is assigned entry and exit conditions.
For a ball to enter (exit) an urn under diffusion, its tags must match the
corresponding condition.
Using Reactants to Define Urn Entry/Exit
Conditions
A new urn is formed each time the system forms
a different kind of reactant with an urn tag.
Reaction Inside a Semi-permeable Membrane
Distributed control results from the control of diffusion
provided by boundaries, combined with the control of
reactions provided by tags (signals).
Urn Model of
Coupled Membrane-enclosed Reactions
x = 111111
y = 111000
Hierarchical Membrane-enclosed Reactions
By controlling concentrations, this hierarchy selects and
amplifies a particular sequence of reactions.
Corresponding Hierarchical Urn Model
The suffix on the each urn entry tag specifies the urn(s) from which
incoming balls may be drawn.
Review of the Urn Model of Controlled Diffusion
Steps in executing the model:
(1) A ball is chosen at random from one of the urns.
(2) The match between the ball’s tags and the exit tags of
an urn determines its probability of
leaving the
urn.
(3) The match between the ball’s tags and the entry tags
of the other urns determines its
probability of
entering another urn.
These steps are repeated to obtain the effect of simultaneous diffusion.
Markov Process Corresponding to
the Urn Model of Controlled Diffusion (1)
h = index of the ball type chosen.
x = index of the urn containing the ball
y = index of the target urn.
When a ball h moves from urn x to urn y then
S(t+1) = ( …, nhx(t)-1, …,nky(t)+1, … ),
where nhx(t) is the number of balls of type h in urn x at time t.
Markov Process Corresponding to
the Urn Model of Controlled Diffusion (2)
qhxy = the probability that ball h will move from urn x to urn y,
as determined by the match scores between the tags.
As with the uncontrolled reactions, the state S(t) at time t is given
by the distribution of the balls in the urns.
A Markov matrix D can be used to define this process, where
Duv = qhxyph.
With b diffusion steps followed by c reaction steps, the result is
S(t+b+c) = S(t)DbRc.
Markov Process Corresponding to
the Urn Model of Controlled Diffusion (3)
Because the Markov matrix D is sparse -- most Duv = qhxyph = 0 -S(t+b+c) = S(t)DbRc can be quickly calculated.
By using states with high occupation probabilities as starting points,
Monte Carlo simulations can quickly determine communities of states
– niches – with large exit times.
Simple examples of interactions in reaction networks with
boundaries and signals:
• mass action and unbalanced flows
• counter-current flows
Some themes:
• persistent patterns in flows
• niche formation and specialization
• diversity and increasing complexity
• modules, motifs, and building blocks
Evolution Generates Feedback and Controls
in Complex Adaptive Systems
Three examples from natural systems:
Niche formation: The “devil’s” garden in Peru.
Octopus “joints” and convergent evolution.
Octopus mimicry of snakes and fish.
Evolution can increase interactions, boundaries, and
signals in reaction networks.
Recapitulation
[摘要重述]
Hypotheses:
Local concentrations of resources, induced by feedback and recycling,
provide opportunities for the formation and adaptive radiation of
agents.
This process of agent formation leads to increasing diversity of agents
and progressively larger amounts of resources “tied up” in agents.
Under ‘tranquil[安静的]’ conditions, increasing agent specialization
should be observed.
If any of these hypotheses can be established we will have substantially
increased our understanding of cas.
Outline
Details
A Quick Review of Reaction Networks
(6)
A Rule-based Version of a Reaction Network
t1##...# & t2##...# => t3##...#
A Quick Review of Reaction Networks
(8)
Formalisms for Reaction Nets
Difference Equations
x(t+1) = x(t) – rx(t)y(t) + r*u(t)v(t),
where x, y. … are concentrations and r, r*, … are reaction rates.
Urn Model
Billiard Ball [撞球] Mechanics (Markov Process)
Rule-based Signal Processing
a1 a 2 a 3 … a k & b1b2… bk T
a 1 …a k b 1 …b k
Urn Models of Tag-based Reactions (1)