A Hybrid Symbolic-Numerical
Method for Determining Model
Structure
Diana Cole, NCSE, University of Kent
Rémi Choquet, Centre d'Ecologie Fonctionnelle et Evolutive
Ben Hubbard, NCSE, University of Kent
Introduction – Example Capture-Recapture
Herring Gulls (Larus argentatus) capture-recapture data for 1983 to
1986 (Lebreton, et al 1995)
78 83
Numbers Released: 𝑹 = 123 84
111 85 Recapture yr
Yr released
83
Numbers Recaptured:
84
85
86
67
4
2
84 𝑵 = 0
103 3
85
0
0
91
Introduction – Example Capture-Recapture
67
4
2
𝑵 = 0 103 3
0
0
91
78
𝑹 = 123
111
i – probability a bird survives from occasion i to i+1
pi – probability a bird is recaptured on occasion i
 = [1, 2, 3, p2, p3, p4 ]
recapture probabilities
𝜙1 𝑝2 𝜙1 (1 − 𝑝2 )𝜙2 𝑝3 𝜙1 (1 − 𝑝2 )𝜙2 (1 − 𝑝3 )𝜙3 𝑝4
𝜙2 𝑝3
𝜙2 (1 − 𝑝3 )𝜙3 𝑝4
𝑸= 0
0
0
𝜙3 𝑝4
3
3
3
𝑁𝑖𝑗
𝑄𝑖𝑗
𝐿=
𝑖=1 𝑗=𝑖
3
1−
𝑖=1
𝑅𝑖 − 3𝑗=𝑖 𝑁𝑖𝑗
𝑗=𝑖
𝑄𝑖𝑗
Can only ever estimate 3 p4 - model is parameter redundant or
non-identifiable.
Introduction
• In some models it is not possible to estimate all the
parameters. This is termed parameter redundant / nonidentifiable.
• A model is parameter redundant if it can be reparameterised
in terms of a smaller number of parameters.
• Capture-recapture example:  = [1, 2, 3, p2, p3, p4 ]
R = [1, 2, p2, p3,  ]  = 3 p4
• Parameter redundancy can be due to the model (extrinsic) or
the data (intrinsic).
• Sometimes it is obvious that a model is parameter redundant
(e.g. capture-recapture example), but in more complex
models it is not necessarily obvious.
Symbolic Method
• Symbolic methods can be used to detect parameter redundancy
in less obvious cases (see for example Catchpole and Morgan,
1997, Cole et al, 2010).
• Firstly an exhaustive summary is required, 𝜿. An exhaustive
summary is a vector of parameter combinations that uniquely
define the model, e.g. recapture probabilities, Q.
3
3
3
𝑁𝑖𝑗
𝑄𝑖𝑗
𝐿=
𝑖=1 𝑗=𝑖
3
1−
𝑖=1
𝑅𝑖 − 3𝑗=𝑖 𝑁𝑖𝑗
𝑗=𝑖
𝑄𝑖𝑗
• Let 𝜽 denote a vector of the p parameters.
• We then form a derivative matrix,
𝜕𝜿
𝑫=
.
𝜕𝜽
Symbolic Method
•
•
•
•
𝜕𝜿
𝑫=
𝜕𝜽
Then calculate the rank, r, of 𝑫.
When r = p, model is full rank; we can estimate all
parameters.
When r < p, model is parameter redundant with deficiency
d=p–r.
In parameter redundant models we can also find a set of r
estimable parameter combinations by solving 𝜶𝑗 𝑫𝑻 = 𝟎 then
𝜕𝑓
𝒑
𝛼
𝒊=𝟏 𝑖𝑗 𝜕𝜃𝑖
al, 2010).
= 0, 𝑗 = 1, … 𝑑 (Catchpole et al, 1998 or Cole et
Problems with the Symbolic Method
• In more complex models the derivative matrix is structurally
too complex. Computer runs out of memory calculating the
rank.
• Examples:
Bio-kinetic compartment
model of sludge respiration
Douchain et al (2007)
Cole et al (2010)
Wandering Albatross
Striped Sea Bass
Multi-state models for sea birds
Tag-return models for fish
Hunter and Caswell (2009)
Jiang et al (2007)
Cole (2012)
Cole and Morgan (2010)
• How do you proceed?
– Numerically – can give the wrong results.
– Symbolically – involves extending the theory and finding
simpler exhaustive summaries (Cole et al, 2010). However
this method is complex.
– Hybrid Symbolic-Numeric Method.
Hybrid-Symbolic Numeric Method
• Calculate the derivative matrix,
𝜕𝜿
𝑫=
,
𝜕𝜽
symbolically.
• Evaluate 𝑫 at a random point 𝜽𝑘 to give 𝑫𝑘 .
• Calculate 𝑟𝑘 the rank of 𝑫𝑘 .
• Repeat for 5 random points model, then 𝑟 = max 𝑟𝑘 .
• If the model is parameter redundant for any 𝑫𝑘 with 𝑟𝑘 = 𝑟
solve 𝜶𝑘 𝑫𝑇𝑘 = 0. The zeros in 𝜶𝑘 indicate positions of
parameters that can be estimated.
Example Capture-Recapture
•  = [1, 2, 3, p2, p3, p4 ]
Example – multi-site capture-recapture model
• The capture-recapture models can be extended to studies
with multiple site (Brownie et al, 1993).
• Example Canada Geese in 3 different geographical regions T=6 years.
• Geese tend to return to the same site – memory model.
(𝑡)
• Initial state probabilities:𝜋𝑗
𝑡
𝑡
𝑡
𝑡
for 𝑗 = 1,2 & 𝑡 = 1, … 6 (𝜋3 = 1 − 𝜋1 − 𝜋2 )
𝑡
• Transition probabilities: 𝜙∗𝑖𝑗 for 𝑖, 𝑗 = 1,2,3 & 𝑡 = 1, … , 5 and 𝜙𝑖𝑗𝑘 for 𝑖, 𝑗, 𝑘 =
1,2,3 & 𝑡 = 2, … , 5.
𝑡
• Capture probabilities: 𝑝𝑗 for 𝑖 = 1,2,3 , 𝑡 = 2, … , 6. (p = 180 Parameters)
Example – Occupancy Models
• Rather than marking animals, occupancy models looks at whether
or not a species is present at a particular site.
• Parameters: 𝜓 − site is occupied, 𝑝 – species is detected.
• Species detected at a site with probability 𝜓𝑝.
• Species not detected at a site with probability
𝜓 1 − 𝑝 + 1 − 𝜓 = 1 − 𝜓𝑝
• Basic model is parameter redundant, so a robust design was
developed, so that several surveys are conducted each season at
each site, and assumed 𝜓 is the same for each survey.
• More complex models consider multiple sites and interactions
between species.
• These models are not parameter redundant, but this assumes that
every possible combination of occupied and unoccupied is
observed. However parameter redundancy can be caused by the
data (intrinsic parameter redundancy).
Example – Occupancy models
• Monitoring of amphibians in the Yellowstone and Grand Teton
National Parks, USA (Gould et al, 2012).
• Two species: Columbian Spotted Frogs and Boreal Chorus Frogs.
• 𝜓 occupancy probabilities, 𝑝 detection probabilities.
• (s) dependence on site, (t) dependence of time, ∙ dependent
on neither site nor time.
Model
𝜓 ∙ 𝑝 ∙
𝜓 𝑠 𝑝 ∙
𝜓 ∙ 𝑝 𝑠
𝜓 𝑡 𝑝 𝑡
𝜓 𝑡, 𝑠 𝑝 ∙
𝜓 𝑡, 𝑠 𝑝 𝑡
𝜓 𝑡, 𝑠 𝑝 𝑡, 𝑠
Rank
20
65
35
59
161
176
236
Deficiency No. pars
0
20
0
65
0
35
0
59
17
178
17
193
67
303
Example - Bio-kinetic compartment model of
sludge respiration
• Non-linear compartment models can be used to describe the
activated sludge-process (Dochain et al, 1995).
• The exogenous oxygen uptake, U, depends on the bio-degredation of
two substrates, S.
• Parameters: 𝜃 = [𝑌1 , 𝑆1 0 , 𝜇𝑚𝑎𝑥1 , 𝐾𝑚1 , 𝑌2 , 𝑆2 0 , 𝜇𝑚𝑎𝑥2 , 𝐾𝑚2 , 𝑋]
where Yi is the fraction of the pollutant Si which is not oxidised
but converted into a new biocatalyst, X. The parameters max1
and max2 are rate constants, Km1 and Km2 are affinity constants.
• From extended symbolic method estimable parameter combinations:
𝜇𝑚𝑎𝑥𝑖 𝑋 1 − 𝑌𝑖
, 𝐾𝑀𝑖 + 𝑆𝑖 0 1 − 𝑌𝑖 , 𝑆𝑖 (0)(1 − 𝑌𝑖 )
𝑌𝑖
Conclusion and future work
• The hybrid method can be used to find how many parameters can be
estimated in a model.
• Hybrid method is much simpler to use than extended symbolic method.
• Can be added to standard software packages. For ecological models it is
available in M-surge and E-surge.
• It can quickly give results about whether a particular data set is parameter
redundant, even for several hundred parameters.
• However it currently is only applicable to a given number of years of data
(ecological models) or substrates (sludge model). In the symbolic method
there is an extension theorem that allows general results to be developed.
Expanding the hybrid method to include the extension theorem is future
work.
• In the parameter redundant model the hybrid method can currently only
determine which of the original parameters are identifiable. Constraints
needed to give an identifiable model can only be obtained by trial and
error. The symbolic method can also give estimable parameter
combinations.
References
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Hybrid Numeric-Symbolic Method:
Choquet, R. and Cole, D.J. (2012) A Hybrid Symbolic-Numerical Method for Determining
Model Structure. Mathematical Biosciences, 236, p117.
Symbolic Method:
Cole, D.J., Morgan, B.J.T., Titterington, D.M. (2010) Mathematical Biosciences, 228, p16.
Cole, D.J., Morgan, B.J.T. (2010), JABES, 15, p431.
Catchpole, E. A., Morgan, B. J. T (1997) Biometrika, 84, p187.
Catchpole, E. A., Morgan, B.J.T., Freeman, S. N. (1998) Biometrika, 85, p42.
Cole, D.J. (2012) Journal of Ornithology , 152, p305.
Other:
Brownie, C. Hines, J., Nichols, J. et al (1993) Capture–recapture, Biometrics, 49, p1173.
Dochain, D, Vanrolleghem, P.A., Van Dale, M. (1995) Water Research, 29, p2571.
Gould, W. R., Patla, D. A., Daley, R., et al. (2012). Wetlands, 32, p379.
Hunter, C., Caswell, H. (2009) Environmental and Ecological Statistics vol 3, p. 797.
Jiang, H.H., Pollock, K.H., Brownie, C. et al, (2007), JABES, 12, p 177
Lebreton, J. Morgan, B. J. T., Pradel R. and Freeman, S. N. (1995) Biometrics, 51, p1418.