OpenStax-CNX module: m38655
1
Takens' Embedding Theorem
∗
Han Lun Yap
This work is produced by OpenStax-CNX and licensed under the
Creative Commons Attribution License 3.0
†
Abstract
Takens' embedding theorem tells us that information about the hidden states of a dynamical system
can be preserved in a time series output.
Indeed, a variety of algorithms for tasks such as time series
prediction and attractor dimension estimation exploit Takens' result. The goal of this module is thus to
detail and discuss Takens' result.
1 Introduction
Takens' embedding theorem tells us that information about the hidden states of a dynamical system can be
preserved in a time series output. Indeed, a variety of algorithms for tasks such as time series prediction and
attractor dimension estimation exploit Takens' result. The goal of this module is thus to detail and discuss
Takens' result.
2 Dynamical Systems
Suppose we have a dynamical system having states
x (t) ∈ RN
that evolve through a dierential equation:
(1)
ẋ = Ψ (x) ,
where
Ψ : RN → RN
represents the vector eld of the dynamical system. Here, we assume
Ψ
is a smooth
function, meaning that the vector eld changes smoothly with the location of the system states, and we are
submanifold M ⊂ RN . To be able
x (t) ∈ M at a given time t, dene the ow functionG : M × R → M by
interested in systems that are conned to a
states
to easily talk about system
(2)
x (t0 + T ) = G (x (t0 ) , T ) ,
for some time
T ∈ R. G
cannot be explicitly calculated but is related to the vector eld
d
dt G (x (t0 ) , t)
Ψ
by
As we are interested in systems that are sampled uniformly in time with some sampling time
time-T mapGT : M → M by
x (t0 + T ) = GT (x (t0 )) ,
∗ Version
1.2: Jul 15, 2011 10:26 am -0500
† http://creativecommons.org/licenses/by/3.0/
http://cnx.org/content/m38655/1.2/
(3)
= Ψ (G (x (t0 ) , t)) .
T,
dene the
(4)
OpenStax-CNX module: m38655
and thus for any
2
k ∈ N,
x (t0 + kT ) = GkT (x (t0 )) = GT ◦ GT ◦ · · · ◦ GT (x (t0 )) .
|
{z
}
k
(5)
times
3 Takens' Embedding
Sometimes, through either ignorance of the variables in the system state vector or technological limitations
on sensor technologies, experimentalists only get to see a one-dimensional time series
Φ : RN → R
is a smooth
observation or measurement function.
states are known (i.e., we have knowledge of
time series measurements by the
system state
x (t)
Ψ(1)),
Kalman lter
1
[3].
y (t) = Φ (x (t)),
the system states can be recovered over time from the
But without this knowledge, can information about the
be retained in this time series data
y (t)?
Remarkably, Takens' theorem [5] answers this in the positive. Takens rst dened the
map
with
M
F(Φ,G−Ts ) : M → RM by stacking M previous
with sampling time Ts up into a vector:
delays
uniformly in time
F (x (t))
delay coordinate
of the time series
y (t)
sampled
T
[y (t) , y (t − Ts ) , · · · , y (t − (M − 1) Ts )]
=
Φ (x (t)) , Φ ◦ G−Ts (x (t)) , · · · , Φ ◦
When it is clear, we will drop the subscripts
M ⊂ RN
2
entries
F(Φ,G−Ts ) (x (t))
=
=
ambient space
where
Note that when the evolution of the system
(Φ, G−Ts )
from
where the system state resides to a
−1
GM
−Ts
F(Φ,G−Ts ) . F
(6)
T
(x (t)) .
is thus a mapping from the
reconstruction space RM
formed with the time
M, Takens showed that given
certain conditions on the time-(−Ts ) map, G−Ts , the delay coordinate map F is an embedding of M onto
the reconstruction space for almost every choice of measurement function Φ. The following theorem, rst
series measurement.
When a dynamical system is conned to a manifold
described in [5], gives the full details:
Theorem 3.1 (Takens' Embedding Theorem) Let
M
K
M > 2K
be a compact manifold of dimension
suppose we have a dynamical system dened by (1) that is conned on this manifold. Let
and
and
suppose:
1. the periodic points of
2.
G−Ts
G−Ts
Then the observation functions
and dense subset of
1 We
with periods less than or equal to
2K
are nite in number, and
has distinct eigenvalues on any such periodic points.
Φ
for which the delay coordinate map
F (6)
is an embedding form an open
C 2 (M, R).
further require that the system and observation functions are linear, and that the system state x (t) and measurements
y (t) sampled uniformly in time ( and are corrupted by white Gaussian noise).
2 We can also consider future entries of the time series but usually for the sake of causality, we shall use only previous entries
unless otherwise stated.
http://cnx.org/content/m38655/1.2/
OpenStax-CNX module: m38655
Figure 1:
3
Pictorial description of Takens' embedding using delay coordinate maps of time series data.
Figure 1 illustrates the process of forming a delay coordinate map
F
is consequently an embedding of the system manifold
all functions
F
is a
f : Ω1 → Ω2
whose
one-to-one immersion.
M.
F
with
M = 3 delays and portrays how
C k (Ω1 , Ω2 ) refers the space of
In the theorem
k -th derivative is continuous. By an embedding, we mean that the operator
F being one-to-one means that distinct system states are not mapped
First,
to the same point in the reconstruction space. Second, an immersion means that the dierential operator
at any point
x
in the state space,
almost every ; basically the
Dx F ,
is itself a one-to-one map. The theorem make clear the meaning of
good observation functions form an open and dense subset of
C 2 (M, R).
M is a limit
Let us give an example of why the extra conditions on periodic points are necessary. Suppose
cycle (or more simply a circle). The conditions of the theorem for this manifold dictates that all periodic
points of M, at least of period 1, be nite in number. Now suppose for an unfortunate choice of Ts , we have
G−Ts (x) = x for all x ∈ M, meaning that the set of all periodic points of period 1 is the whole manifold
itself. Then no matter what observation function or however many delays M we pick, the delay coordinate
T
map will have the form F (x) = [Φ (x) , Φ (x) , · · · , Φ (x)] for any x ∈ M. This implies that the limit cycle
will be mapped onto a line in the reconstruction space, thus violating the one-to-one property.
Summing up, this theorem says that if the conditions about periodic points on the dynamical systems are
Φ' ∈ C 2 (M, R), there will be a function Φ in an arbitrarily small neighborhood
around Φ such that the delay coordinate map F(Φ,G−T ) is an embedding. This means that for a large
s
class of observation functions Φ, F preserves the topology of M, therefore information about M can be
fullled, then for any function
'
retained in the time series (measurements) output! Indeed by preserving the topology of the manifold in the
reconstruction space, many properties of the manifold and the dynamical system are retained, including its
dimensionality and its lyapunov exponents - just to name a few.
http://cnx.org/content/m38655/1.2/
OpenStax-CNX module: m38655
4
4 Related Work
Independently but at nearly the same time as Takens' original work, Aeyels [1] looked at the same problem
from a control theory standpoint. He showed that the delay coordinate map is related to the observability
criteria and that given any system in
of observation function
h
N
dimensions (not just one conned to a manifold), a generic choice
guarantees that the system is observable as long as
M > 2N .
Some interesting dynamical systems have system states that converge onto a
some transient time (instead of residing on a manifold).
M > 2K ,
where
K
is now the
again an embedding of
of
almost every
M
chaotic attractor M
after
Sauer et al. [4] showed that if we again picked
box-counting dimension of the attractor, then the delay coordinate map F
for almost every observation function
Φ.
is
The authors also sharpened the notion
probability one.3
delay coordinate maps composed of non-uniformly
to correspond more closely to the heuristic notion of having
More recently, Huke and Broomhead [2] showed that
sampled time series data (instead of just uniformly sampled ones considered here) can also be embeddings of
the system manifold. In particular, this means that delay coordinate maps formed using
interspike intervals
coming from spiking neuron models can be an embedding of the dynamical system that is the input to the
neuron model.
References
[1] D. Aeyels. Generic observability of dierentiable systems. SIAM J. Control and Optimization, 19:595,
1981.
[2] J. P. Huke and D. S. Broomhead. Embedding theorems for non-uniformly sampled dynamical systems.
Nonlinearity, 20(September):2205, 2007.
[3] R. E. Kalman. A new approach to linear ltering and prediction problems. J. Basic Eng., 82(1):358211;45,
1960.
[4] Tim Sauer, James A. Yorke, and Martin Casdagli. Embedology. J. Stat. Physics, 65(3-4):5798211;616,
November 1991.
[5] Floris Takens. Detecting strange attractors in turbulence. In Dynamical Syst. and Turbulence, Warwick
1980, volume 898 of Lecture Notes in Mathematics, page 3668211;381. Springer Berlin / Heidelberg, 1981.
Φ.
3 More
specically, the authors showed that F is an embedding for a prevalent subset of the set of all observation functions
http://cnx.org/content/m38655/1.2/