Injectivity Analysis using Interval Analysis.
Application to Structural Identifiability ?
Sébastien Lagrange a , Nicolas Delanoue a , Luc Jaulin b
a
b
Laboratoire Ingénierie des Systèmes Automatisés (LISA), 62 avenue Notre Dame du Lac, 49000 Angers, France
Laboratoire Développement Technologies Nouvelles (DTN), 2 rue François Verny, 29806 Brest Cedex 09, France.
Abstract
This paper presents a new numerical algorithm based on interval analysis able to prove that a differentiable function f : A ⊂ Rn → Rn
is injective. This algorithm also performs a partition of the domain A in subsets Ai where, for all x ∈ Ai , the cardinality of f −1 (f (x))
is constant. In the context of parameter estimation, we show how this algorithm provides an efficient and numerical method to study the
structural identifiability of parametric models.
Key words: Injectivity, parametric system, structural identifiability, interval analysis.
1
1.1
Introduction
Structural identifiability
The notion of (structural) identifiability is the question of
whether one can hope uniquely to estimate the parameters
of a model from experimental data that can be collected.
This question is particulary relevant since the parameters
have a concrete meaning and whenever decisions have to be
taken on the basis of their numerical values [2]. Consider
the model structure M(.) for which the vector of parameters
p belongs to a prior set P and denote by M(p̃) = M(p)
the identical input-output behavior generated by the models M(p̃) and M(p). The structural identifiability of M(.)
depends to the number of solutions of M(p̃) = M(p).
Define Sp = {p̃ ∈ P | M(p) = M(p̃)}. If, for almost
any p ∈ P, Sp is a singleton, the vector of parameters p
is structurally globally identifiable (s.g.i); if, for almost any
p ∈ P, Sp is finite or countable, p is structurally locally
identifiable (s.l.i). Otherwise, p is structurally unidentifiable
[10]. When testing model for identifiability, the standard approach is in two steps.
During the first step, equations of the form f (p̃) = f (p)
equivalent to M(p̃) = M(p) are established. According to
? This paper was not presented to any IFAC meeting. Corresponding author S. Lagrange. Tel. +33-2-41-22-65-72; Fax +33-2-4122-65-61.
Email addresses: [email protected]
(Sébastien Lagrange), [email protected]
(Nicolas Delanoue), [email protected] (Luc Jaulin).
Preprint submitted to Automatica
the structure M(·), various methods can be used to obtain
f [6,8–10]. In the following, we consider that the function
f is available.
During the second step, the set of all solutions p̃ of f (p̃) =
f (p) is sought for any p. When each component of f is
polynomial in the entries of p; elimination theory can be
used to put this set of polynomial equations into a triangular form. Then, it becomes possible (at least in principle) to
solve f (p̃) = f (p) by solving a succession of polynomial
equations in a single unknown and thus to obtain the set
of all the parameter vectors p̃ that satisfy these equations.
However, formal approaches may turn out to be unsatisfactory for different reasons: i) If Equation f (p̃) = f (p) is not
polynomial, formal calculus often fails to reach a solution.
ii) The complexity of the formal manipulations is high. iii)
The number of solutions p̃ depends on the value of p. It is
then difficult to reach a structural conclusion. The following
example illustrates the points i) and iii). It will be treated
with the new approach advocated in this paper.
Example 1 Consider the model f (p) = p cos(p) where the
parameter p belongs to P = [−3, 3] and no input. For any
(p̃, p) ∈ P 2 , M(p̃) = M(p) iff p̃ cos p̃ = p cos p. Let a and
b be the real numbers defined in Figure 1. The set of feasible values for p̃ is a triple for p ∈] − a, −b[∪] − b, b[∪]b, a[,
a pair for p ∈ {−a, −b, b, a} and a singleton for p ∈
[−3; −a[∪]a, 3] (see Figure 1).
21 April 2008
Fig. 2. Graph µ[−a,a] : [−3, 3] → N∪{+∞} of f (x) = x cos(x).
The graph of f has been superposed as dotted-lines.
Fig. 1. The grey zone corresponds to the values of x for which
the equation f (x) = f (x̃) has more than one solution.
1.2
that h(x, y) = 0. Thus, the proposition can be rewritten
∀y ∈ [y] , ∃!x ∈ [x] , h (x, y) = 0. To perform this test, we
generalize the interval Newton method [4,5,7] and define an
extension of the unicity operators. This new result will be exploited in the next section in order to show that a function f :
Rn → Rn satifies ∀y ∈ [y] , ∃!x ∈ [x] such that f (x) = y
(by setting h(x, y) = f (x) − y).
Let us recall the definition of an (interval) unicity operator.
Consider a function f : Rn → Rn and [x] ⊂ Rn . An operator N : Rn → {true, false} is a unicity operator of f if
N (f , [x]) ⇒ ∃!x ∈ [x] , f (x) = 0. The standard (interval)
unicity operator is the unicity operator of Newton [1,4].
We extend the unicity operator defined for functions from
Rn to Rn to functions defined from Rn × Rm to Rn . These
extended operators will be named generalized unicity operators. Consider a function h : Rn × Rm → Rn and two
boxes [x] ⊂ Rn , [y] ⊂ Rm . For y ∈ [y], one defines hy :
Rn 3 x 7→ hy (x) = h(x, y) ∈ Rn . Now, let N (hy , [x]) be
a unicity operator of hy then we get N (hy , [x]) ⇒ ∃!x ∈
[x] , hy (x) = 0. Since N (h, [x] , y) , N (hy , [x]), N is a
unicity operator of the function h at the point y. Therefore
N (h, [x] , [y]) ⇒ ∀y ∈ [y] , ∃!x ∈ [x] , h (x, y) = 0.
Contents of the paper
The purpose of the paper is to present an alternative route,
based on guaranteed numerical computation, able to study
the structural identifiability. Concisely, consider a differentiable function f defined from A ⊂ Rn to Rn , we define the
injectivity function of f by
(
A → N ∪ {+∞}
µ:
x → | {x̃ ∈ A | f (x̃) = f (x)} |
where | · | denotes the cardinality of a set. The function µ
associates to each vector x the number of solutions x̃ of the
equation f (x̃) = f (x). This paper proposes a new algorithm
able to enclose the function µ between two functions µ−
and µ+ such that µ− (x) ≤ µ(x) ≤ µ+ (x). This algorithm
is relevant for identifiability analysis of parametric models
as previously illustrated. It can decompose the parametric
space into different domains. In each of this domains the
number of feasible vectors of parameters is constant (and
known). To our knowledge, it does not exist any numerical
and guaranteed approaches to enclose µ. This paper presents
the first attempt in that direction. Note that Braems and al.
have presented in [3] an approximated (numerical) method
to test whether the function µ is equal to one.
The paper is organized as follows. Section 2 presents interval analysis and some possibilities of this tool. Section
3 defines the injectivity function restricted to a domain and
points out its main properties. By a combination of interval analysis and the properties of the injectivity function, an
effective algorithm is built in Section 4. In Section 5, an illustrative example shows the efficiency of the algorithm to
test models for structural identifiability. A C++ solver called
IAVIA is made available. Note that the proposed technique
only works for low dimensional problems since the proposed
algorithm takes an exponential amount of time in terms of
the dimension.
2
Definition 1 An operator N : Rn × Rm → {true, f alse}
is a generalized unicity operator of h : Rn × Rm → Rn if
N (h, [x] , [y]) ⇒ ∀y ∈ [y] , ∃!x ∈ [x] , h (x, y) = 0.
3
Injectivity function
This section presents some properties of µ. This will motivate the implementation of the algorithm presented in the
next section.
Let [x̃] be a box included in [x], we define the injectivity
function (of f ) restricted to [x̃], denoted µ[x̃] , by
(
[x] → N ∪ {+∞}
µ[x̃] :
x → |{x̃ ∈ [x̃] | f (x̃) = f (x)}| .
µ[x̃] (x) is the number of solutions x̃ of the equation f (x̃) =
f (x) that belongs to [x̃] (see example of Figure 2). Note
that, since we only consider the injectivity function of f , we
omit to specify it in the following.
Interval Newton Method
The purpose of this section is to give a sufficient condition
to verify the proposition : ∀y ∈ [y], ∃x ∈ [x] such that
(
h(x, y) = 0
∀x0 ∈ [x], (h(x0 , y) = 0 ⇒ x = x0 )
where h : Rn × Rm → Rn is a differentiable function and
[x] , [y] are two boxes of Rn and Rm . The implication means
that for all y there exists a unique x (denoted ∃!x) such
Proposition 1 Consider a function f : [x] ⊂ Rn → Rn
and a collection [x̃1 ], . . . , [x̃q ] of boxes of [x]. For I =
P
{1, . . . , q}, we get µ S [x̃i ] (x) =
(−1)|J|−1 µ T [x̃i ] (x).
i∈I
2
J⊂I
i∈J
Algorithm 1 Injectivity_Enclosure
Input: f , [x] and [x1 ] ⊂ [x].
Output: µ− and µ+ such that µ− ≤ µ[x] ([x1 ]) ≤ µ+
1: Initialization : Lstack := {[x]}, [µ− , µ+ ] = [0, 0].
2: while Lstack 6= ∅ do
3:
Pop Lstack into [w].
4:
if [f ] ([w]) ∩ [f ] ([x1 ]) 6= ∅ then
5:
if width([w]) > width([x1 ]) then
6:
Bisect [w] into [w1 ] and [w2 ].
7:
Stack [w1 ] and [w2 ] in Lstack
8:
else
9:
Push [w] in Lw .
10: end while
{here, f −1 (f ([x1 ])) ⊂ ∪i [wi ], [wi ]inLw }
11: Build a partition Lx̃ from Lw {see Figure 3}
12: for i = 1 to size of(Lx̃ ) do
13:
if [Df ([x̃i ])] is full rank then
14:
if N (h, [x̃i ], f ([x1 ])) then
15:
µ− = µ− + 1
16:
else
17:
Return [0, +∞] {Enclosure failed}
18: end for
19: Return [µ− , size of (Lx̃ )]
Fig. 3. Illustration of Step 11 with Lw = {[w1 ], . . . , [w6 ]}. The
[wi ]’boxes are organized in order to obtain the two disjoint (and
larger) boxes [x̃1 ] and [x̃2 ] stored in Lx̃ .
Proof. Trivial (use the inclusion-exclusion principle).
Let us introduce the following theorem which gives a sufficient condition to enclose µ over a box [x1 ] ⊂ [x].
Theorem 1 If f : [x] ⊂ Rn → Rn is differentiable (denote
Df its Jacobian matrix), N is a generalized unicity operator
of h(x, y) , f (x) − y, [x1 ] ⊂ [x] and {[x̃i ]}i∈{1,...,q} be a
set of boxes which satisfies
i) i 6= j ⇒ [x̃i ] ∩ [x̃j ] = ∅,
S
ii) f −1 (f ([x1 ])) ⊂ i [x̃i ],
iii) ∀i ∈ {1, . . . , q 0 }, N (h, [x̃i ], f ([x1 ])) is true,
iv) ∀i ∈ {q 0 + 1, . . . , q}, [Df ([x̃i ])] is full rank
where q 0 ∈ N, 1 ≤ q 0 ≤ q, then q 0 ≤ µ[x] ([x1 ]) ≤ q.
Proof. µ[x] ([x1 ]) = µS
+ µ[x]\S
(see
S
Proposition 1). Since ii), we get µ[x] ([x1 ]) = µ [x̃i ] ([x1 ]).
i
Pq
According to i), µ[x] ([x1 ]) = i=1 µ[x̃i ] ([x1 ]) (see Proposition 1). Trivially, according to iii), one has ∀i ∈ {1, . . . , q 0 },
µ[x̃i ] ([x1 ]) = 1. And, according to iv), one has ∀i ∈ {q 0 +
1, . . . , q}, µ[x̃i ] ([x1 ]) ≤ 1 (this result can be proved by contradiction considering the Mean Value Theorem). Therefore
Theorem 1 is proved.
4
i
[x̃i ] ([x1 ])
i
[x̃i ] ([x1 ])
Fig. 4. Intervals from dark grey to light grey are respectively the
domain where the injectivity function µ[−3,3] is equal to 3 and
1. The black intervals correspond to the undeterminate domains
(where the enclosure of µ[−3,3] is µ− = 0 and µ+ = +∞). The
white intervals (close to black ones) correspond to the indeterminate domain where the function µ[−3,3] belongs to [1, 3].
Algorithm IAVIA
This section presents the algorithm IAVIA (Injectivity Analysis Via Interval Analysis) whose aim is to enclose µ. IAVIA
can be decomposed in two distinct sub-algorithms : Algorithm 1 exploits Theorem 1 in order to return an interval
[µ− , µ+ ] which encloses µ[x] over a box [x1 ] ⊂ [x]. Algorithm 2 divides the initial box [x] into a paving {[xi ]}i
such that, for all i, Algorithm 1 succeeds in obtaining an
enclosure of µ over the [xi ]’boxes. Different domains are
obtained : an indeterminate domain composed of the boxes
of width lower than ε (stored in U ) and domains where the
µ is perfectly known are stored in the list S.
also excludes the possibilities to proved that, for example,
f (x) = x3 is injective over [−a, a], a ∈ R.
Example 2 Consider the function presented in Example 1.
With a condition ε = 10−3 and after 10 seconds on a Pentium III 1.8GHz, the solver IAVIA returns the boundaries
presented in Figure 4.
5
Remark 1 Note that if Df (x) is not full rank, then every
boxes [x] such that x ∈ [x] verify [Df ([x])] is not full rank.
Therefore, the condition at Step 13 of Algorithm 1 is never
satisfied and the enclosure process fails. This implies that
{x | Df (x) is not full rank } is included in U . The proposed algorithm is not able to evaluate the injectivity (or
the structural identifiability) for the values of x which are
local maximizer/minimizer of f (x). This limitation of IAVIA
Numerical test for structural identifiability
This section gives an illustrative example which shows the
efficiency of the (numerical) algorithm IAVIA to test models
for structural identifiability. Consider the ad hoc parametric model M (·) defined by the following state equations :
(
ẋ(t) = [(1 − p2 ) p1 c1 − p2 s1 − 2p2 ] x(t) + u(t)
y(t) = [p1 (1 + s1 − p2 s1 ) + p2 c1 ] x(t)
3
Algorithm 2 IAVIA
Input: f a C 1 function and [x] the initial box.
Output: A list S that contains sub-boxes of [x] and the
corresponding bounds of the injectivity function.
1: Initialization : L := {[x]}, S = ∅, U = ∅.
2: while L 6= ∅ do
3:
Pull [w] in L.
4:
[µ− , µ+ ] =Injectivity_Enclosure(f , [x] , [w])
5:
if width([w]) > ε then
6:
if µ− = µ+ then
7:
Push ([w] , µ− ) in S
8:
else
9:
Bisect [w] into [w1 ] and [w2 ]
10:
Push [w1 ] and [w2 ] in L
11:
else
12:
Push ([w] , [µ− , µ+ ]) in U
13: end while
IAVIA
Ljung and Glad
Braems
Termination
no
yes
no
Guaranteed
yes
yes
no
Numeric
yes
no
yes
Constrained
domains
yes
no
yes
limitations
few parameters
algebraic models
Fig. 6. Comparaison between different approaches
µ(x) = |{x̃ ∈ [x] | f (x̃) = f (x)}|. To our knowledge, it
did not exist any numerical method able to perform this
enclosure. Note that, in case of functions f : R → R and
f : R2 → R2 , the solver IAVIA developed in C++ is made
available at http://www.istia.univ-angers.fr/
~lagrange/. In the context of parameter estimation, we
have shown that the proposed algorithm is able to test models
for structural identifiability. Table 6 gives comparaison with
methods available in the literature. The method proposed
by Ljung and Glad in [6] uses differential algebra to test
global structural identifiability. However, it is impossible to
take into account constraints on parameters (the natural non
negativity of a (physical) parameter cannot be imposed).
Our method is numerical and can prove s.g.i. with respect
to a constrained parameter space. Moreover, IAVIA is able
to decompose the parametric space into subspaces Pi such
that s.g.i. with respect to Pi is proved.
Fig. 5. Enclosure of µ[p] obtained by IAVIA. In light gray, the value
of the parameters for which the system is structurally identifiable
(µ+ = µ− = 1). The two dark gray zones correspond to the value
of the parameters for which the system is locally identifiable, 2 or 3
parameter vectors are possible (µ+ = µ− = 2 or µ+ = µ− = 3).
In the white (tiny) domains, µ is proved to belong [1, 3].
References
[1] G. Alefeld. Inclusion methods for systems of nonlinear equations the interval newton method and modifications. In Proceedings of the
IMACS-GAMM International Workshop on Validated Computation,
pages 7–26, Oldenburg, Germany, August 1993.
where c1 = cos(p1 ), s1 = sin(p1 ) and the vector of param1
eters p = (p1 , p2 )T belongs to [10, 26] × [0, 10
]. By the use
of Laplace transform approach, M(p) = M(p̃) translates
into f (p) = f (p̃) where the components of f are the coefficients ofÃthe transfer matrix of M:
!
(1 − p2 ) p1 cos p1 − p2 sin p1 − 2p2
.
f (p) =
p1 (1 + sin p1 − p2 sin p1 ) + p2 cos p1
The analysis of structural identifiability of M amounts to
count the number of solutions of f (p) = f (p̃), for all
p ∈ [p]. Or equivalently, it consists in studying the injectivity function µ[p] (·) of f . If µ[p] (p) = 1, p is s.g.i; if µ[p] (p)
is bounded, p is s.l.i. Otherwise, p is unidentifiable. After
60 min, the results of the enclosure of µ[p] are depicted in
Figure 5. Note that in the context of structural identifiability test, the existence of a box [x1 ] such that µ−
[x] ([x1 ]) ≥ 2
could stop Algorithm IAVIA and no structural identifiability
should be proved. It only consists to change the condition of
Step 6 of IAVIA by "If µ− ≥ 2". In this case, the structural
unidentifiability of model M is proved in less than 1 sec.
6
Method
[2] F. Berthier, J. P. Diard, L. Pronzato, and E. Walter. Identifiability
and distinguishability concepts in electrochemistry. Automatica,
32(7):973–984, July 1996.
[3] I. Braems, L. Jaulin, M. Kieffer, and E. Walter. Guaranteed numerical
alternatives to structural identifiability testing. In Proceedings of the
40th IEEE Conference on Decision and Control., volume 4, pages
3122–3127, Orlando, December 2001.
[4] E. Hansen and R.I. Greenberg. An interval newton method. APPL.
MATH. COMPUT., 12(2-3):89–98, 1983.
[5] L. Jaulin, M. Kieffer, D. Didrit, and E. Walter. Applied Interval
Analysis. Springer, 2001.
[6] L. Ljung and T. Glad. On global identifiability of arbitrary model
parametrizations. Automatica, 30:265–276, 1994.
[7] A. Neumaier. Interval Methods for Systems of Equations. Cambridge
Univ. Press, 1990.
[8] M. P. Saccomani, S. Audoly, G. Bellu, L. D’Angio, , and C. Cobelli.
Global identifiability of nonlinear models parameters. In IFAC Symp.
on System Identification SYSID’97, pages 219–224, Japan, 1997.
[9] S. Vajda, K. R. Godfrey, and H. Rabitz. Similarity transformation
approach to identifiability analysis of nonlinear comportemental
models. Math. Biosc., pages 217–248, 1989.
Conclusion
[10] E. Walter and L. Pronzato. Identification of parametric models from
experimental data. Springer, 1997.
In this paper, we proposed a new numerical and guaranteed
method to enclose the injectivity function of f defined by
4