Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
rsos.royalsocietypublishing.org
Research
Cite this article: Hu Y, Wang S, Ma N,
Hingley-Wilson SM, Rocco A, McFadden J,
Tang HL. 2017 Trajectory energy minimization
for cell growth tracking and genealogy
analysis. R. Soc. open sci. 4: 170207.
http://dx.doi.org/10.1098/rsos.170207
Trajectory energy
minimization for cell
growth tracking and
genealogy analysis
Yin Hu1 , Su Wang1 , Nan Ma2 ,
Suzanne M. Hingley-Wilson2 , Andrea Rocco2 ,
Johnjoe McFadden2 and Hongying Lilian Tang1
1 Department of Computer Science, Faculty of Engineering and Physical Sciences,
and 2 Department of Microbial and Cellular Sciences, School of Biosciences and
Medicine, Faculty of Health and Medical Sciences, University of Surrey, Guildford,
Surrey GU2 7XH, UK
Received: 8 March 2017
Accepted: 24 April 2017
Subject Category:
Computer science
Subject Areas:
molecular biology/computer vision/
image processing
Keywords:
time-lapse image analysis, cell segmentation
and tracking, Escherichia coli, level set
framework, trajectory energy minimization
Author for correspondence:
Yin Hu
e-mail: [email protected]
YH, 0000-0003-2258-9787
Cell growth experiments with a microfluidic device produce
large-scale time-lapse image data, which contain important
information on cell growth and patterns in their genealogy.
To extract such information, we propose a scheme to segment
and track bacterial cells automatically. In contrast with
most published approaches, which often split segmentation
and tracking into two independent procedures, we focus
on designing an algorithm that describes cell properties
evolving between consecutive frames by feeding segmentation
and tracking results from one frame to the next one. The
cell boundaries are extracted by minimizing the distance
regularized level set evolution (DRLSE) model. Each individual
cell was identified and tracked by identifying cell septum
and membrane as well as developing a trajectory energy
minimization function along time-lapse series. Experiments
show that by applying this scheme, cell growth and division
can be measured automatically. The results show the efficiency
of the approach when testing on different datasets while
comparing with other existing algorithms. The proposed
approach demonstrates great potential for large-scale bacterial
cell growth analysis.
1. Introduction
Elucidating the mechanisms underlying cell growth, division and
phenotypic variation is a significant goal for biological research.
Phenotypes of bacterial cells such as size, growth, division, death
and genealogy can be observed by a human observer using
2017 The Authors. Published by the Royal Society under the terms of the Creative Commons
Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted
use, provided the original author and source are credited.
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
2
................................................
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
laboratory techniques. The study of the phenotypes based on a large population can provide valuable
knowledge in development and variability of cellular response to drugs [1,2]. For instance, bacterial
cell growth imaging experiments can provide massive amounts of time-lapse image data of cell growth
by confocal microscopy, with far more information than a human observer can digest. Therefore, it has
become evident that automated segmentation and tracking are indispensable to handle and examine the
large number of image data captured in even a single experiment with high efficiency, consistency and
completeness.
Automated analysis of the cell growth and division process is challenging for many reasons. Firstly,
the low signal–noise ratio and the large variability of image objects can make the membrane and
septum of cells difficult to identify in the image. Secondly, the cell population densities are exponentially
increasing due to cell elongating and dividing; this makes it hard to accurately recognize the actual
shapes of cells, especially in spatially crowded clusters of cells. In addition, the challenges of cell tracking
lie in the fact that the cell dynamics can cause changes of cellular morphologies, such as elongation,
division, death and entering/leaving the field of view. Maintaining the consistency of segmentation and
tracking results between the consecutive frames is a key objective task for many published approaches.
In previous work, to deal with the challenges discussed above, some strategies for image-based
studies of cellular growth and division have been introduced, which fall into the three following basic
categories. First, digital images can be manually analysed by a human with expert knowledge and
skills [3]. This approach is tedious, time consuming and has a low degree of reproducibility. Second,
time-lapse microscope images are fully automatically analysed by using image processing methods, such
as segmentation and tracking algorithms. However, existing algorithms are not robust enough for this
purpose and generate inaccurate segmentation and tracking results [4]. Third, to address the problems
mentioned above, time-lapse images can be automatically processed and followed by manual editing
and correcting. Schnitzcells [5] and MicrobeTracker [6] are two popular systems that can do quantitative
analysis of fluorescent time-lapse images of living cells. However, such systems are laborious and not
reproducible.
A comprehensive survey on the latest computational automatic analysis and software tools has been
undertaken in [7]. They can be classified into two groups: tracking by detection and tracking by matching.
In the first framework, cells are detected in each frame and then associations between segmented cells in
consecutive sequences are established by certain criteria. This category of methods is based on the ‘first
segment, then track’ scheme, as seen in [8–10]. A comparison of different cell segmentation methods
has been presented in [11], where gradient features [8], cell properties [12], intensity [13,14], region
accumulation and level set [15] are discussed. In addition, a review of object tracking approaches
has been presented in [16], and includes sequential Monte Carlo methods [17], joint probabilistic
data association filtering [18], multiple hypothesis tracking [19,20], integer programming [14], dynamic
programming [21] or coupled minimum-cost flow tracking [22]. They are applied to determine the most
likely cell correspondence between frames. One of the major merits for this category is its computational
efficiency of segmentation stage. When only one cell is present in the field of view, the trajectory can be
plausibly formed by connecting the cell location over time, and it is easier to recover from tracking failure.
In addition, detection and association steps are the mutual independence, which allows straightforward
tracking of new cells entering the field of view [23]. However, it is difficult to identify the real number
of cells if cell densities are high, a large number of cell divisions occur, or cells enter and exit the field
of view [24]. Moreover, their results are not always consistent between frames since their detection and
tracking steps are mutually independent.
To avoid these problems, in the second framework, segmentation and tracking procedures are
performed simultaneously. This is based on fitting a model to cells and on employing the result in the
current frame as the initial points for segmentation in the next frame. This is to evolve the contours of
the cells, represented either parametrically [25–27] or implicitly [28–33] using a velocity term defined by
the content of the ‘target’ frame (such as gradient features, intra- and inter-region heterogeneity, shape
or topology). They use morphological and behavioural clues in the model to handle the topologically
flexible behaviour of cells. In addition, they try to address the changing number of cells because of cell
division and dying, and cells entering or exiting the frame. The major drawback is that small errors in
localization can accumulate [34]. Combining both frameworks together, Li et al. [30] proposed a complex
cell tracking system that integrates a fast level set framework with a local association step.
Although these methods show good performance, they still have difficulties in segmenting and
tracking precisely in crowded cell clusters in low-contrast images without fully identifying and recording
the cell division process. To achieve these, the segmentation and tracking results should be consistent
between frames. However, this is a major challenge for most of published methods. In this work, we
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
propose an effective method to detect and track bacterial cells in large time-lapse series generated from
various experiments. There are three major contributions:
2. Edge-based active contour model in distance regularized level
set evolution
In this section, we briefly review the formulation and implementation of the DRLSE proposed by Li
et al. [35]. This method introduced a more general variational level set [36] with a distance regularization
term and an external energy term that drives the motion of the zero level set towards expected locations.
This model has become popular in extracting medical image contours, such as magnetic resonance
imaging (MRI) images of heart [37] and computerized tomography (CT) images of brain haemorrhage
[38], due to its efficiency and ability to eliminate the need for reinitialization.
Let I be an image on domain Ω and φ : Ω → is a level set function, an energy function E(φ) can be
formulated as:
E(φ) = μRp + λLg + αAg
=μ
p(|∇φ|) dx + λ
gδ(φ)(|∇φ|) dx + α
gH(−φ) dx,
Ω
Ω
Ω
(2.1)
where p is an energy density function p : [0, ∞) ∈ , δ is the Dirac delta function [39], and H is the
Heaviside function [40]. μ > 0, λ > 0 and α ∈ are the coefficients of these three terms. ∇ is the gradient
operator. The edge indicator function g is defined by
g
1
,
1 + |∇Gσ ∗ I|2
(2.2)
where Gσ is a Gaussian kernel with a standard deviation σ . The convolution in equation (2.2) is used to
smooth the image to reduce the surplus noise. This function g usually takes smaller values at expected
object boundaries than at other locations.
— In equation (2.1), the first term is a penalty term Rp to maintain the contour to evolve near the
signed distance function, i.e. |∇φ| = 1. Li et al. introduced the function p as:
⎧ 1
⎪
(1 − cos(2π s)), if s ≤ 1
⎪
⎨
(2π )2
(2.3)
p(s) =
⎪
⎪
⎩ 1 (s − 1)2 ,
if s > 1.
2
................................................
In the following sections, we describe our segmentation and tracking approach and compare it with that
of MicrobeTracker and MAMLE on phase contrast images from previously published experiments and our
own datasets. These published datasets were obtained growing as largely isolated cells or micro-colonies
on agarose pads, which can provide high-quality images. On the contrary, our datasets show densely
growing cells in the microfluidics system, which are lower-quality images but provide more information
to understand their growth patterns through their life cycles. In this work, we focus on segmenting and
tracking cells which are from the microfluidics system.
The rest of the paper is structured as follows. Section 2 briefly reviews the distance regularized
level set evolution (DRLSE) and its minimization. Section 3 introduces the materials and the time-lapse
datasets analysed in our experiments. Section 4 describes in detail our proposed scheme. Section 5
gives the experimental evaluations and discussions. The paper concludes with suggestions for future
work in §6.
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
— first, the profile information of cell septum and membrane is used to identify the cell division
process and segment touched cells as cells exponentially grow in numbers;
— second, the global trajectory energy minimization function is developed to efficiently track cells
during cell elongation and division, even if the features of cells such as length and area are
changing all the time. This process can minimize the accumulated errors; and
— finally, combining local region homogeneity with the internal properties of the evolved contours
(cell profile changes) can largely maintain a coherent segmentation and tracking results between
the consecutive frames.
3
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
4
1h
1.5 h
2h
(scale bar: 10 mm)
Figure 1. Time-lapse frame samples of our image data.
— The second term Lg in equation (2.1) calculates the line integral of the function g along the zero
level set of φ. By parametrizing the zero level set of φ as a contour C : [0, 1] → Ω, the energy can
1
be expressed as a line integral 0 g(C(s))|C (s)| ds [41]. The energy is minimized when the zero
level set φ is located at the desired edge of object.
— The third term Ag in equation (2.1) calculates a weighted area of the region Ωφ− x : φ(x) < 0.
This energy is used to speed up the motion of the zero level set in the evolution process, which
is necessary when the initial contour is set far away from the expected boundaries of the object.
The steepest descent process for minimization of the total energy functional is the following
gradient flow:
∇φ
∂φ
= μ div(dp (|∇φ|)∇φ) + λδ(φ) div g
+ αgδ(φ),
(2.4)
∂t
|∇φ|
where div(·) is the divergence operator and dp is a function defined by dp (s) p (s)/s.
The DRLSE allows the use of more general functions as the initial contour; however, the position of
the initial contour needs to be decided. If the initial contour is close to the region to be segmented, only a
small number of iterations are needed to move the zero level set to the expected boundary of the object.
In addition, this model can deal with the image with weak boundaries successfully.
3. Experiment set-ups and materials
In this section, we introduce the time-lapse dataset of Escherichia coli (E. coli) cells analysed in our
experiments. Figure 1 shows some sample frames from the E. coli dataset.
3.1. Bacterial strains and culturing
The E. coli wild-type (MG1655) strain was obtained from Balaban et al. [42]. Figure 2 demonstrates the
sequence of cell division. The formation of the septum becomes increasingly apparent during the cell
growth and division. LB Lennox (Sigma Aldrich) was used for all liquid growth media, plus technical
agar no. 3 (Sigma Aldrich) for plates. Ampicillin (Sigma Aldrich) was used at a final concentration of
100 µg ml−1 . For batch culture kill curves, aliquots were taken from an overnight culture and incubated
with ampicillin for up to 48 h, at room temperature, with colony forming units measured at time 0 and
then at named intervals.
3.2. Microfluidics platform
Table 1 gives a brief summary of acquisition set-ups and image data properties. Bacterial cells in late
log phase were filtered five times through a 24 gauge needle and then loaded into a pre-warmed (37◦ C)
microfluidic system (CellASIC ONIX Microfluidic Platform with B04A Microfluidic Bacteria Plate, with
pressurized height of 0.7 µm). Media flow was controlled via the CellASIC system (which provides media
exchange in less than 30 s) at a flow rate of approximately 10 µl h−1 . Imaging was performed under a
Nikon confocal microscope (Nikon A1M on Eclipse Ti-E) equipped with an environmental chamber,
motorized stage and perfect focus system.
Automated multi-area imaging (eight fields of view for whole growth chamber) was carried out using
a 40× air objective lens (Nikon Apo λ) with a numerical aperture of 0.95 (which gave a sub-cellular
resolution of approx. 0.32 µm). The ‘perfect focus’ system locked the cells in objective lens focal plane for
the whole period of experiments with a resolution of ±25 nm. For the growth period images were taken
at regular intervals of approximately 69 s (depending on experiment) to limit photo bleaching and laser
damage, and imaging carried out for around 8 to 10 generations.
................................................
0.5 h
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
t=0h
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
5
cell elongates
cell septum
septum forms completely
cells separate
Figure 2. A diagram of the sequence of E. coli cell division.
Table 1. A summary of acquisition set-ups and image data properties for E. coli cells analysed in our experiments.
cell type
Escherichia coli
imaging system
Nikon A1M on Eclipse Ti-E
objective lens
40× air objective lens (Nikon Apo λ) with a
numerical aperture of 0.95
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
calibration (µm px−1 )
0.15–0.16
time step (seconds)
69
number of frames
120–140
number of series
6
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
4. Proposed cell image analysis framework
Our proposed segmentation feed tracking scheme involves two steps. First, a Gaussian filtering is
applied to each frame to reduce the amount of noise and enhance cell membrane and septum structures.
Second, the cell regions are detected by integrating the DRLSE and object tracking frameworks
simultaneously. The initial cell models generated in the previous frame are evolved in time to feed
the corresponding cells in the consequent frame. With the cell elongation and division over time,
clustered cells need to be split, and the mother and daughter cell relationships need to be maintained.
To achieve these requirements, the DRLSE framework is integrated with a cell septum and membrane
indication function that exploits a characteristic intensity profile across cells. In the rest of this section, we
further explain each individual step and describe how to deal with new entering cells and unnecessary
computations of the DRLSE framework.
4.1. Image preprocessing
Since the proposed method particularly relies on the local intensity distribution to locate cell membrane
and septum, it is important to reduce the effect of noise. In our implementation, we apply convolution
with a Gaussian mask with a standard deviation of 1. Figure 3 shows, comparing with other kernels, that
this amount of smoothing suppresses noise sufficiently while preserving true membrane and septum
of cells.
4.2. First frame segmentation
One of the common ways is to apply a level set model to segment a single set of objects and then the level
set contour is split into multiple sub-contours based on boundaries of individual objects. This approach
................................................
cell begins to divide
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
cell membrane
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
(a)
(b)
(c)
(d)
(b)
(c)
R0
Figure 4. The segmentation process of the first frame. (a) Original input image with an isolated cell. (b) Preprocessed image
superimposed by an initial contour R0 . (c) Processed image superimposed by the final contour by minimizing the DRLSE framework.
works well in segmenting objects which have clear boundary or contrast against background. However,
in our work, we need to deal with images with weak cell boundaries and with cells clustered together.
To fulfil these requirements, we used one level set to evolve independently for each cell in the first frame
and coevolution of a set of non-overlapping level sets for the consequent frames. The initial contours
of each cell are detected and used as the ‘seeds’. These seeds move outwardly and finally stop near the
boundary of the cells.
Since cells are not crowded together in the first frame, their rough boundaries are easily obtained by
using simple segmentation methods (such as adaptive thresholding). These boundaries are regarded as
the initial contours R0 (figure 4b) prepared for the DRLSE framework. The final contours are detected
by minimizing equation (2.1) in this framework. In places where a steady state is reached, small foreign
objects (such as dust and bubbles) are eliminated from the final binary mask based on a predefined
threshold rs . The remainder are considered as the cells to be tracked in the following frames.
4.3. Object tracking
One of the crucial aims of each tracking method is to preserve the identity of every tracked target
over time. Since cell displacements between two successive frames are not too large, many reported
approaches [28–33] used the previous cell segmentations as initial objects for the subsequent frames.
However, these approaches might yield undesired results. For example, when two isolated cells in a
certain frame become a cluster in the next one, it is essential to prevent their contours from merging.
More importantly, to construct a cell genealogy tree, the properties of relations between mother and
daughter cells need to be maintained and consistent. For instance, when one mother cell forms two
isolated daughter cells in one frame, it is necessary to preserve these two isolated cell identities in the
following frames. To fulfil these requirements, the DRLSE framework has been integrated with a set of
indication functions.
4.3.1. Object indication function
Maška et al. [31] assumed that the image can be split into a possibly disconnected background region
and many possibly disconnected disjoint foreground objects. However, when a possibly disconnected
disjoint foreground object contains different cells or other objects, it might yield undesired results. We
have largely addressed this limitation by using cell boundary information. Assume we have obtained
in the previous frame t − 1, and it will be used to extract the corresponding cell region
one cell Bt−1
1
Bt1 in the current frame t by making use of the overlapped region V (figure 5). The intersection V of
and Bt1 should be first obtained and then grown to the boundary of Bt1 (figure 5g). To obtain the
Bt−1
1
................................................
(a)
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
Figure 3. (a) Original image. Smoothed images after applying: (b) Gaussian filter with standard deviation of 1. (c) Gaussian filter with
standard deviation of 2. (d) 5 × 5 median filter.
6
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
(a)
(b)
frame t – 1
(c)
frame t
(d)
(e)
zoomed-in frame t zoomed-in frame t
(f)
membrance gradient profile
V
Bt1–1
0
Figure 5. Object tracking process: (a) Segmented bacterial cell Bt−1
1 in previous frame t − 1. (b) Current frame t is superimposed by the
t−1
segmentation result Bt−1
1 . (c) Split B1 into small windows in current frame t. (d) In current frame t, classify these windows into two
categories: with (small windows in black) or without information of cells’ boundaries (small windows in blue) by using intensity gradient
profiles along perpendicular direction visualized in a red line. The two points in yellow indicate the edge of cell. (e) An intensity gradient
t−1
t
profile. (f ) The blue region is the intersection V of Bt−1
1 and B1 , and the pink region is the intersection of B1 and the background. (g) The
intersection V grows to the final boundary of B1 in frame t.
overlapped region V, each initial object (previous segmented cell region Bt−1
1 ) is split into several small
windows (w is the size of the window) (figure 5c), whereas the regions of different initial objects cannot
merge together. By assigning different positive labels to individual cells, two previously separated cells
can be distinguished when they touch each other. Each sub-window can merge later in time based on a
homogeneity criterion:
— In the first step, these windows are classified in two different categories: with or without
information on cells’ boundaries (figure 5d). Since these boundaries have high-intensity
gradients, this information is used to identify those small regions which locate on cells’
boundaries (figure 5e). The windows with boundary information are eliminated to obtain
multiple isolated regions. These regions might include three possible types: the intersection V,
t−1
the intersections of Bt−1
1 and other cells, and the intersections of B1 and background (figure 5f ).
— In the second step, the DRLSE coevolution framework is applied to these remaining subwindows to obtain their corresponding contours. This procedure might form some undesired
contours. For instance, many contours are generated by intersection of Bt−1
1 and the background.
Furthermore, many different cells might merge together as some sub-window regions cannot be
fully isolated for a single given cell. A rod-shape and size criterion can be used to eliminate these
undesired contours. We also proposed a global energy function to deal with initial cell regions
expanding to different cells.
— Finally, the remaining contours are only merged within same category and same cell.
4.3.2. Cell septum and membrane indication function
For each obtained cell region, a cell septum indication function is proposed to efficiently identify and
record mother and daughter cells’ relationship to construct a cell trajectory and genealogy, which is often
a challenging task through many published tracking approaches [7].
As a cell is growing and starting to divide, the formation of the septum becomes increasingly apparent
(figure 6). The cell septum indication function exploits a characteristic intensity profile across the septum
of cells to identify the stages of cell growth (red line in second column of figure 6), such as elongation and
division stage. To handle cases of dividing cells which shared the same septum (figure 6b), we propose
a criterion to automatically separate connected cells at an optimal point (figure 7). In our experiments,
the local minimum of the profile was considered to be the separating point (figure 7c) if the value of
the local minimum lm is lower than a threshold rm . As shown in figure 7d, these two separated cells are
regarded as two new daughter cells from their mother cell. For each frame, this threshold is denoted
as rm = μseptum + τ σseptum , where μseptum and σseptum are the mean and the standard deviation of the
obtained cells’ boundary regions in the current frame, respectively. τ will be discussed in §5.
In addition, the width of cells is regarded as another indicator to identify the membranes in contact
when a cluster of cells get crowded. If the cell width is larger than a threshold rw , the cells will be split
by using the same strategy discussed above.
4.3.3. Global trajectory energy indication function
The initial regions of one cell might expand within different cells which yields undesired results. To avoid
this issue, inspired by [43], we proposed a global trajectory energy function which depends on all cells in
7
................................................
0.02
frame t
Bt1
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
gradient
0.04
(g)
frame t
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
profiles of
septum
sketches
8
................................................
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
125
intensity
(a)
original input
image
t = 34th
frame
100
75
(b)
intensity
125
shared same
septum
t = 40th
frame
100
75
(c)
intensity
125
t = 45th
frame
100
75
(d)
intensity
125
t = 50th
frame
100
75
Figure 6. Cross-section profiles of different growing stages at different time. Red lines are the major axis of the cells, and green lines
indicate the septum threshold rm . (a) Profile of cell when t = 34 (single mother growing cell). (b) Profile of cell when t = 40 (two
daughter cells share same septum). (c) Profile of cell when t = 45 (dividing into two connected daughter cells). (d) Profile of cell when
t = 50 (separated into two daughter cells).
(a)
(b)
(c)
(d)
lm
intensity
125
100
lm
rm
75
Figure 7. Automated separation of two connected cells. (a) Original input image data. (b) Sketch image shows local minimum (red dot),
major axis (red line) and cutting line (white line). (c) Profile of the major axis with local minimum lm (red dot) and threshold rm (green
line). (d) Segmentation result.
all frames. Owing to cell elongation and division, cell properties such as length and area are not constant
and cannot be used as strong cues for cell tracking. This energy function is also to address this situation.
To obtain the trajectories of cells, each possible contour discussed above is used to sequentially segment
and track its corresponding cells. Thus, each individual cell has different trajectories over an entire cell
growing cycle from birth to division. In addition, when the cell septum indication function that defines
one cell divides into two new cells, two new separate trajectories are generated.
Let the state vector X contain positions of all cells in all frames. Xtn is the location of cell n at frame t.
F and N define the total number of frames and cells, respectively. The temporal length of trajectory of
cell n is denoted by F(n) := dn − bn + 1, where bn and dn are the first birth and final division frames,
respectively. Our energy function is made up of three terms:
E(X) = βEvel (X) + γ Eexc (X) + ηEreg (X),
(4.1)
where β, γ and η are weights that control the relative influence of each term.
The first term Evel (X) is a motion model based on a standard constant velocity assumption, which
is used to reject impossible motion patterns and address ambiguities between crossing trajectories,
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
defined by:
9
− 2Xt+1
n
2
+ Xt+2
n .
(4.2)
n=1 t=bn
The second term Eexc (X) is to penalize configurations where cells come too close to each other. In
addition, this model helps to enforce unique data association since each segmented result only can be
assigned to one trajectory. It is defined as:
Eexc (X) =
N(t)
F r2w
t=1 n,j=n
Xtn − Xtj 2
.
(4.3)
where the scalar rw accounts for the minimum average width of a complete cell.
The last term Ereg (X) is to drive the optimization towards a solution for the situation in which
occasionally cells keep elongating without division causing longer trajectories N, which is defined by:
Ereg (X) = N +
N
1
,
F(n)
(4.4)
n=1
All possible trajectory energies of each cell are calculated, and the minimum of the energies is regarded
as the optimal trajectory.
4.4. Capturing new entering cells
It is necessary to pay attention to new cells entering into the field of view. They have not been identified in
the previous frames, which possibly lead to false segmentation and tracking results at a particular frame.
After segmenting and tracking the current frame without taking new entering cells into consideration,
the first frame segmentation scheme is used to capture them from non-extracted cell areas. For each new
entering cell, a new DRLSE function and a new trajectory is created and evolved.
4.5. Stopping criterion
So far, we have assumed that the minimization of the DRLSE model stops when a steady state is reached.
For every iterative algorithm, it is essential to specify a suitable stopping criterion to avoid unnecessary
computations when the algorithm converges. A fixed number of iterations is often the first choice,
which assumes that the desired contour will be achieved after evolving the curve for a large number of
iterations. However, the desired contour might have been achieved long before the iteration procedure
finishes. Thus, considering the computational complexity of the level set implementations for large-scale
time-lapse cell segmentations, this unnecessarily increases the execution time. Also, we must be careful
so as not to terminate the iteration procedure prematurely.
In the case of our implementation, we use a general stopping criterion based on the changes of the
three terms (Rp , Lg and Ag ) in equation (2.1). When the steady state is obtained, it is obvious that these
three terms remain almost constant. Based on these observations, the difference between two successive
iterations i is defined by:
2
i
i−1 2
i
i−1 2
i = Rip − Ri−1
p + Lg − Lg + Ag − Ag ,
i ≥ 1,
(4.5)
where i − 1 and i are two successive iterations. In the DRLSE framework, this criterion stops the curve
evolution process when i ≤ K is satisfied (figure 8). K could be used to improve the computation times
as the final contour often does not change too much during the last iterations. If K is set to zero, the
iteration stops when a steady state is reached. In our experiment, the choice of K will be discussed in §5.2.
5. Experimental results and evaluation
Our proposed scheme was implemented using Matlab on a computer equipped with Intel Core i5
processor running at 2.2 GHz. Firstly, we describe the approaches to evaluate both segmentation and
tracking performance, including an empirical evaluation, Dice coefficient, the multiple object tracking
precision (MOTP) and the multiple object tracking accuracy (MOTA) [44]. Then, we discuss the optimal
parameter settings for our scheme based on our datasets. We then compare our proposed scheme with the
published tracking packages MicrobeTracker and MAMLE [45]. Meanwhile, we discuss the experimental
results and highlight the differences between our proposed scheme and other developed systems.
................................................
Xtn
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
Evel (X) =
N d
n −2
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
15
10
5
K
0
0
30
60
iteration i
90
Figure 8. The difference between two successive iterations (i ) during the evolution of the DRLSE model. When i ≤ K is satisfied,
the minimization process of this model will stop.
5.1. Evaluation approaches
5.1.1. Datasets
We evaluated the segmentation and tracking performance on three E. coli datasets: one from the
MicrobeTracker example data [6], one from Schnitzcells [5], and one from our own dataset. All datasets
were manually annotated by human experts as the ground truth. The two public datasets were
obtained growing as largely isolated cells or micro-colonies on agarose pads, which can provide
high-quality images comparing to our dataset. However, our dataset gave densely growing cells in
the microfluidics system, which provides information when studying large-scale individual cells to
understand their growth patterns through their life cycles. This dataset contains six different experiments
with approximately 4000 cells. Each experiment is 180 min in length. It considers cell count as an objective
metric for evaluation.
5.1.2. Segmentation evaluation
According to the specifications of different approaches, cells which are undergoing division are classified
as a single cell or as two independent cells. To have a fair comparison of the evaluated methods, these
kinds of classifications are not considered as wrong segmentations. The results are classified into four
categories:
—
—
—
—
true positive (TP): a cell is segmented properly;
over-segmentation: a cell is split into more than one object;
under-segmentation: more than one cell is merged as a single cell; and
false negative (FN): a cell is not detected.
Since false positive is a small fraction of detections (less than 0.1%) and its overall contribution is
insignificant, this result is discarded from the evaluation.
To quantitatively evaluate the segmentation results, we compute the Dice coefficient between the
manually annotated region (A) and the automatic segmented region (B). The Dice coefficient is computed
as 2|A ∩ B|/(|A| + |B|).
5.1.3. Multiple object tracking evaluation
Cell tracking can be regarded as a multiple object tracking algorithm. Although there are no standard
evaluation approaches or datasets for multi-target tracking algorithms, the general tracking performance
still can be evaluated by MOTP and MOTA. Assuming that for each time t, a multiple object tracker
outputs a set of hypotheses {h1 , . . . , hm } for a set of visible objects {o1 , . . . , on }, MOTP judges how well
exact positions of objects are estimated by computing the Euclidean distance disit between every object
oi and its corresponding hypothesis.
i
i,t dist
,
(5.1)
MOTP = t ct
where ct is the number of matched object–hypothesis pairs found for time t.
................................................
Di
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
10
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
5.2. Parameter settings
Our proposed method has 13 parameters that influence its overall performance. They can be divided
into four groups based on their purposes. We used 10% of our datasets for deciding the parameters of
our proposed scheme. The following gives details of various experimental set-ups as well as additional
experiments to justify some of our design choices.
In the preprocessing stage, to reduce the amount of noise and preserve the rod-like structures, a
Gaussian filter (σ = 1) was performed in all experiments. The choice of σ relies on the width of the
rod-like structures to be preserved. In addition, larger values of σ will smooth out the cell’s membrane
and septum.
The next group of parameters rs , τ , rw and w in §4.1 and 4.3 indicates the quantitative properties of
segmented and tracked cells. They can be obtained from analysed time-lapse datasets:
— The parameter rs is defined at 3 µm according to the minimum average size of a complete cell
from our datasets. All objects touching the image boundary were removed.
— As a cell is growing and starting to divide, the formation of the septum becomes increasingly
apparent. The parameter τ is defined at 1.2. When the local minimum of the septum profile is
lower than the septum threshold, the connected cell is separated at the local minimum point.
— Furthermore, clustering of cells often appeared in our time-lapse datasets. Therefore, the
parameter rw was fixed at 1.2 µm to identify this situation.
— During cell tracking procedure, to ensure the sub-window includes the information of
membrane, the size of sub-window should be larger than the size of membrane. The parameter
w is defined at 0.6 µm pixels.
These parameters are converted into different values in pixel level according to the different
magnification level of the microscope.
Thirdly, the most important parameters are the weights of the DRLSE framework. These are
parameters λ, μ and α in equation (2.4), and the time step t for implementation. Li et al. [35] mentioned
that the DRLSE model is not sensitive to the choice of λ and μ, which can be fixed for most of applications.
The parameter α needs to be tuned for different types of images. In our work, α is set to a negative
value to expand the contour. A non-zero α gives additional external force to drive the motion of contour.
For a large absolute value of α, the contour easily passes through the object boundary in images with
weak object boundaries. Therefore, we examined different parameter configurations: λ = 1, 5, . . . , 300,
μ = 0.01, 0.02, . . . , 0.50, α = 0, −1, . . . , −5 and t = 1, 2, . . . , 10. In the stopping criterion of the DRLSE
model, the choice of K can speed up the computation to obtain the optimal segmentations. To obtain
the optimal parameters for DRLSE model equation (2.4), the optimization scheme was performed
on the evaluation dataset. Table 2 lists the top five of the parameters based on the Dice coefficient
and execution time. In the following experiments, we set time step t = 1, μ = 0.2, λ = 5, α = −3
and K = 0.005.
The last group of parameters characterizes the trajectory energy function used in our proposed
scheme. There are three parameters β, γ and η in equation (4.1). They were examined on different
configurations from 0.1 to 1.0. Table 3 lists the top five of these parameters based on the MOTP and
MOTA. These parameters were set to {0.02, 0.5, 0.5} in all our experiments.
5.3. Results and discussion
5.3.1. Segmentation
We compared the segmentation results of our proposed scheme with MicrobeTracker and MAMLE. It
is necessary to note that neither MicrobeTracker nor MAMLE were developed for dealing with the
................................................
where gt is the number of objects present at time t. In detail, the MOTA can be regarded as three error
ratios: the ratio of false negatives FN = t FNt / t gt , the ratio of false positives FP = t FPt / t gt , and
the ratio of mismatches MM = t MMt / t gt .
11
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
In addition, the MOTA shows how many mistakes the tracker made in terms of misses or false
negatives (FNt ), false positives (FPt ) and mismatches (MMt ).
(FNt + FPt + MMt )
MOTA = t
,
(5.2)
t gt
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
data 2
data 3
12
our proposed
scheme
MAMLE
MicrobeTracker
................................................
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
original image
data 1
Figure 9. Zoomed-in segmentation results: data 1 is from the MicrobeTracker example data, data 2 is from Schnitzcells and data 3 is from
our own dataset. The first row is the original images, the second row is the segmentation results of MicrobeTracker, the third row is the
segmentation results of MAMLE, the last row is the segmentation results of our proposed method.
Table 2. Top five results of parameters in DRLSE model based on Dice coefficient and execution time.
i , μ, λ, α, K
1, 0.200, 5, −3, 0.005
Dice coefficient
0.931
time (s)
125
3, 0.067, 5, −1, 0.005
0.925
112
1, 0.200, 4, −3, 0.010
0.871
105
2, 0.100, 5, −3, 0.008
0.851
182
2, 0.100, 3, −2, 0.005
0.805
98
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
Table 3. Top five results of parameters in trajectory energy function based on the multiple object tracking precision (MOTP) and the
multiple object tracking accuracy (MOTA).
β, γ , η
0.02, 0.5, 0.5
MOTP (pixel)
92
MOTA (%)
78.36
0.02, 0.6, 0.6
91
77.56
0.03, 0.5, 0.5
141
69.58
0.01, 0.6, 0.7
155
59.62
0.04, 0.5, 0.7
222
54.94
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
data in microfluidics that we have. Figure 9 shows segmentation results of three different methods
on MicrobeTracker, Schnitzcells and our dataset. These results demonstrated that our proposed scheme
efficiently segments cells out when the septum is not clear and the membrane is too close (see third
high
no. cells (per image)
110 600 (700)
methods
TP
79 632
over-seg. (total %)
10 321 (9.33%)
under-seg. (total %)
9058 (8.19%)
FN (total %)
11 589 (10.48%)
72.00%
89 586
7201 (6.51%)
8210 (7.42%)
5603 (5.07%)
81.00%
Our proposed scheme
95 116
6512 (5.89%)
4647 (4.20%)
4325 (3.91%)
86.00%
.................................................................................................................................................................................................................................................
MAMLE
.................................................................................................................................................................................................................................................
MicrobeTracker
segmentation
accuracy (%)
272
108 801 (401)
83 777
8941 (8.22%)
8201 (7.54%)
7882 (7.24%)
77.00%
92 481
6402 (5.88%)
5004 (4.59%)
4914 (4.52%)
85.00%
Our proposed scheme
96 833
4200 (3.86%)
4012 (3.68%)
3756 (3.45%)
89.00%
.................................................................................................................................................................................................................................................
MAMLE
.................................................................................................................................................................................................................................................
MicrobeTracker
512
40 960 (80)
34 816
2248 (5.49%)
1923 (4.69%)
1973 (4.82%)
85.00%
37 683
1351 (3.29%)
879 (2.15%)
1047 (2.56%)
92.00%
Our proposed scheme
38 912
701 (1.71%)
632 (1.54%)
715 (1.75%)
95.00%
.................................................................................................................................................................................................................................................
MAMLE
.................................................................................................................................................................................................................................................
MicrobeTracker
942
260 361 (276)
198 225
21 510 (8.26%)
19 182 (7.37%)
21 444 (8.24%)
76.13%
219 750
14 954 (5.74%)
14 093 (5.41%)
11 564 (4.44%)
84.40%
Our proposed scheme
230 861
11 413 (4.38%)
9291 (3.57%)
8796 (3.38%)
88.67%
.................................................................................................................................................................................................................................................
MAMLE
.................................................................................................................................................................................................................................................
MicrobeTracker
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
................................................
.............................................................................................................................................................................................................................................................................................................................................................................
total
.............................................................................................................................................................................................................................................................................................................................................................................
low
.............................................................................................................................................................................................................................................................................................................................................................................
medium
.............................................................................................................................................................................................................................................................................................................................................................................
no. images
158
density level
Table 4. The segmentation accuracy and generality of the different methods in various density levels. The bold values indicate the best performance for each density level.
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
13
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
(b)
1
6 11 16 21 26 31 36 41 46 51
frames
10
9
8
7
6
5
4
3
2
1
0
(c)
1
6 11 16 21 26 31 36 41 46 51
frames
10
9
8
7
6
5
4
3
2
1
0
14
1
6 11 16 21 26 31 36 41 46 51
frames
Figure 10. The consistency level for the number of extracted cells in sparse and medium crowded frames: (a) extracted cell number from
the MicrobeTracker, (b) extracted cell number from MAMLE and (c) extracted cell number from our own dataset.
Table 5. The proportion of correctly identified cells at optimal Dice coefficient for different methods on three datasets. The bold values
are the best performance for each dataset.
MicrobeTracker
data 1
78.66
data 2
70.29
data 3
63.34
MAMLE
83.51
81.55
78.64
Our proposed scheme
80.95
79.62
87.56
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
column of figure 9). To provide a compelling reason why the other two approaches are not sufficiently
good for the large-scale individual cell studies, we evaluated the consistency level for the number of
extracted cells in figure 10. These two previously published methods often produce non-consistent results
between consecutive frames. The results in table 4 reveal the high segmentation accuracy and generality
of the proposed method in different density levels, comparing to the other two methods. Illustrative
examples from test samples are presented in the third column of figure 9.
Table 5 shows the comparison of the segmentation algorithms based on the proportion of cells
correctly identified at different datasets at optimal Dice coefficient. It shows that the performance of
our proposed scheme is comparable to that of the state-of-the-art algorithms on cells grown on agarose
pads. Furthermore, it has obvious improvements on segmentation results on cells grown on microfluidics
system.
5.3.2. Tracking
To further evaluate the performance of our proposed scheme, we tested its tracking results on a largescale experiment. As Schnitzcells, MAMLE and MicrobeTracker did not provide sample video, we used
our datasets to test. According to the cell density in our dataset, we chose the 100th frame (around fifth
generation) as a threshold to categorize time-lapse images into medium and highly crowded density.
MAMLE did not provide a tracking algorithm, we combined its segmentation algorithm with the tracking
algorithm of Schnitzcells, since MAMLE outperforms Schnitzcells based on segmentation under many
circumstances [46].
Tables 6 and 7 illustrate the tracking performance based on precision and accuracy for datasets with
medium and highly crowded density, respectively. Our proposed scheme has higher MOTA than the
other two published methods in both medium and highly crowded density. In detail, our proposed
trajectory energy minimization function significantly reduces the error ratio of FN, FP and MM, which
are critical tasks in cell tracking systems. Figure 11 shows an illustration of the tracking results of the
proposed method for a single cell. Figure 12 shows the genealogy tree of a single cell.
5.3.3. Cell growth analysis
In this work, we aimed to observe the cell growth and patterns in their genealogy. After segmenting
and tracking, we analysed the size at birth, size at division, division time and elongation rate. Figure 13
reveals cells elongate approximately exponentially at the single-cell level.
................................................
10
9
8
7
6
5
4
3
2
1
0
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
no. cells
(a)
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
15
140
frames
120
100
80
60
s2
s1
40
20
m
0
Figure 12. The genealogy tree of a single cell. There are around 6 generations from this single cell (m). Y-axis shows the number of
frames in this time-lapse video. For instance, the cell (m) has existed in the first frame (frame = 0). After 38 frames, cell m divides into
two daughter cells (s1 and s2 ), and the division time for s1 and s2 are 20 frames and 18 frames, respectively.
Table 6. The performance for our dataset of the group with medium crowded density based on the multiple object tracking precision
(MOTP), the ratio of false negatives (FN), the ratio of false positives (FP), the ratio of mismatches (MM) and the multiple object tracking
accuracy (MOTA). The bold values indicate the best performance on MOTP and MOTA.
MOTP (pixel)
FN (%)
FP (%)
MM (%)
MOTA (%)
Schnitzcells
261
10.91
1.08
18.58
69.43
MicrobeTracker
308
6.56
1.58
10.50
81.36
Our proposed scheme
268
5.83
0.79
8.07
85.31
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
Table 7. The performance for our dataset of the group with highly crowded density. The bold values indicate the best performance on
MOTP and MOTA.
Schnitzcells
MOTP (pixel)
159
FN (%)
22.33
FP (%)
5.83
MM (%)
42.54
MOTA (%)
29.30
MicrobeTracker
181
19.81
Our proposed scheme
168
14.54
2.49
22.50
55.20
2.41
18.95
64.10
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
.........................................................................................................................................................................................................................
................................................
160
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
Figure 11. Tracking results of a single cell from its birth to division by using the proposed method. The white cell in the earliest frame is
the mother of this single cell, while the yellow cells show the changes of this single cell during its growth. The last frame highlights its
daughter cells of the division event shown in purple and white. Red cells are not a part of this single cell.
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
7.5
E. coli data
exponential fit
7.0
16
length (mm)
5.5
5.0
4.5
4.0
3.5
0
5
10
15
20
time (min)
25
30
35
Figure 13. The representative growth curve of a single cell (dark blue circles).
6. Conclusion and future work
In this paper, we proposed an effective scheme to automatically segment and track E. coli cells in
large time-lapse image series. The method incorporates the DRLSE framework with cell septum and
membrane indication function and trajectory energy minimization function, which deal with cell
segmentation and tracking together to perform cell genealogy analysis as well as various measures
related to cell growth. The testing of our method was conducted on two published datasets and one
from our own experiments. It shows that our proposed method produces competitive results, both in
segmentation and tracking. Especially, it could be preferred in large-scale studies focusing on growth
characteristics and genealogy of cells in which as-accurate-as-possible cell segmentation is the most
desirable feature. We further analysed the cell growth over time from the birth to its division, and
measured its size at birth, size at division, division time and elongation rate.
Currently, we are developing deep learning techniques to facilitate the tracking and measurement of
a variety of cells. We are currently applying our proposed segmentation and tracking scheme to other
bacterial cells, such as Mycobacterium smegmatis. Although our method works well on different density
data, further improvement of this scheme will handle over-crowded data where cell shapes are irregular
and not constant between consecutive frames, as the limited space available in microfluidic devices is
quickly filled up by exponentially growing bacterial populations. Furthermore, we would like to speed
up our proposed scheme and integrate sub-pixel precision in segmentation.
Data accessibility. Our data are deposited at Dryad: http://dx.doi.org/10.5061/dryad.0q35m [47].
Authors’ contributions. Y.H. and H.L.T. carried out the design of the study, participated in statistical data analysis and
drafted the manuscript; S.W. conceived of the structure of system; N.M. and S.M.H.-W. carried out the molecular
laboratory work, sequence alignments and collected field data; A.R., J.M. and H.L.T. finalized the manuscript. All
authors gave final approval for publication.
Competing interests. We declare we have no competing interests.
Funding. This work was kindly funded by the Biotechnology and Biological Sciences Research Council (BBSRC,
BB/J002097/1).
Acknowledgements. The authors thank the owners of MicrobeTracker, Schnitzcells and MAMLE for making their data and
systems publicly available.
References
1. Brock A, Chang H, Huang S. 2009 Non-genetic
heterogeneity: a mutation-independent driving
force for the somatic evolution of tumours. Nat. Rev.
Genet. 10, 336–342. (doi:10.1038/nrg2556)
2. Trott J, Hayashi K, Surani A, Madan Babu M,
Martinez-Arias A. 2012 Dissecting ensemble
networks in ES cell populations reveals
micro-heterogeneity underlying pluripotency. Mol.
Biosyst. 8, 744–752. (doi:10.1039/c1mb05398a)
3. Egerton FN. 2006 A history of the ecological
sciences, part 19: Leeuwenhoek’s microscopic
natural history. Bull. Ecol. Soc. Am. 87, 47–58.
(doi:10.1890/0012-9623(2006)87[47:AHOTES]
2.0.CO;2)
4. Prewitt JMS, Mendelsohn ML. 1966 The analysis of
cell images. Ann. NY Acad. Sci. 128, 1035–
1053. (doi:10.1111/j.1749-6632.1965.
tb11715.x)
5. Young JW, Locke JCW, Altinok A, Rosenfeld N,
Bacarian T, Swain PS, Mjolsness E, Elowitz MB. 2012
Measuring single-cell gene expression dynamics in
bacteria using fluorescence time-lapse microscopy.
Nat. Protoc. 7, 80–88. (doi:10.1038/nprot.2011.
432)
6. Sliusarenko O, Heinritz J, Emonet T, Jacobs-Wagner
C. 2011 High-throughput, subpixel precision analysis
of bacterial morphogenesis and intracellular
................................................
6.0
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
6.5
Downloaded from http://rsos.royalsocietypublishing.org/ on June 16, 2017
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34. Kanade T, Yin Z, Bise R, Huh S, Eom S, Sandbothe
MF, Chen M. 2011 Cell image analysis: algorithms,
system and applications. In 2011 IEEE Workshop on
Applications of Computer Vision (WACV),
pp. 374–381. New York, NY: IEEE.
35. Li C, Xu C, Gui C, Fox MD. 2010 Distance
regularized level set evolution and its
application to image segmentation. IEEE Trans.
Image Process. 19, 3243–3254. (doi:10.1109/
TIP.2010.2041414)
36. Osher S, Sethian JA. 1988 Fronts propagating
with curvature-dependent speed: algorithms
based on Hamilton-Jacobi formulations. J. Comput.
Phys. 79, 12–49. (doi:10.1016/0021-9991(88)
90002-2)
37. Jin-qing L, Wei-wei L. 2011 Adaptive medical image
segmentation algorithm combined with DRLSE
model. Proc. Eng. 15, 2634–2638. (doi:10.1016/
j.proeng.2011.08.495)
38. BhanuPrakash KN, Zhou S, Morgan TC, Hanley DF,
Nowinski WL. 2012 Segmentation and
quantification of intra-ventricular/cerebral
hemorrhage in CT scans by modified distance
regularized level set evolution technique. Int. J.
Comput. Assist Radiol. Surg. 7, 785–798.
(doi:10.1007/s11548-012-0670-0)
39. Gomes J, Faugeras O. 2000 Reconciling distance
functions and level sets. J. Vis. Commun. Image
Represent. 11, 209–223. (doi:10.1006/jvci.1999.
0439)
40. Weber M, Blake A, Cipolla R. 2004 Sparse finite
elements for geodesic contours with level-sets. In
Computer Vision-ECCV 2004, pp. 391–404. Berlin,
Germany: Springer.
41. Caselles V, Kimmel R, Sapiro G. 1997 Geodesic active
contours. Int. J. Comput. Vis. 22, 61–79. (doi:10.1023/
A:1007979827043)
42. Balaban NQ, Merrin J, Chait R, Kowalik L, Leibler S.
2004 Bacterial persistence as a phenotypic switch.
Science 305, 1622–1625. (doi:10.1126/science.
1099390)
43. Andriyenko A, Schindler K. 2011 Multi-target
tracking by continuous energy minimization. In
2011 IEEE Conf. on Computer Vision and Pattern
Recognition (CVPR), pp. 1265–1272. New York, NY:
IEEE.
44. Bernardin K, Stiefelhagen R. 2008 Evaluating
multiple object tracking performance: the CLEAR
MOT metrics. J. Image Video Process. 2008, 246309.
(doi:10.1155/2008/246309)
45. Chowdhury S, Kandhavelu M, Yli-Harja O, Ribeiro
AS. 2013 Cell segmentation by multi-resolution
analysis and maximum likelihood estimation
(MAMLE). BMC Bioinform. 14, S8. (doi:10.1186/14712105-14-S10-S8)
46. Sadanandan S, Baltekin O, Magnusson K, Boucharin
A, Ranefall P, Jalden J, Elf J, Wahlby C. 2015
Segmentation and track-analysis in time-lapse
imaging of bacteria. IEEE J. Select. Top. Signal
Process. 10, 174–184. (doi:10.1109/JSTSP.2015.
2491304)
47. Hu Y, Wang S, Ma N, Hingley-Wilson SM,
Rocco A, McFadden J, Tang H. 2017 Data from:
Trajectory energy minimisation for cell growth
tracking and genealogy analysis. Dryad
Digital Repository. (doi:10.5061/dryad.
0q35m)
17
................................................
8.
21.
sequences. IEEE Trans. Pattern Anal. Mach. Intell. 35,
2736–3750. (doi:10.1109/TPAMI.2013.97)
Magnusson KE, Jaldén J. 2012 A batch algorithm
using iterative application of the Viterbi algorithm
to track cells and construct cell lineages. In 9th IEEE
Int. Symp. on Biomedical Imaging (ISBI), 2012,
pp. 382–385. New York, NY: IEEE.
Padfield D, Rittscher J, Roysam B. 2011 Coupled
minimum-cost flow cell tracking for
high-throughput quantitative analysis. Med. Image
Anal. 15, 650–668. (doi:10.1016/j.media.2010.
07.006)
Bise R, Yin Z, Kanade T. 2011 Reliable cell tracking by
global data association. In 2011 IEEE Int. Symp. on
Biomedical Imaging: from Nano to Macro, pp.
1004–1010. New York, NY: IEEE.
Wen Q, Gao J, Luby-Phelps K. 2007 Multiple
interacting subcellular structure tracking by
sequential Monte Carlo method. In IEEE Int. Conf. on
Bioinformatics and Biomedicine, 2007. BIBM 2007,
pp. 437–442. New York, NY: IEEE.
Dufour A, Thibeaux R, Labruyere E, Guillen N,
Olivo-Marin J-C. 2011 3-D active meshes: fast
discrete deformable models for cell tracking in 3-D
time-lapse microscopy. IEEE Trans. Image Process.
20, 1925–1937. (doi:10.1109/TIP.2010.2099125)
Zimmer C, Labruyere E, Meas-Yedid V, Guillen N,
Olivo-Marin J-C. 2002 Segmentation and tracking of
migrating cells in videomicroscopy with parametric
active contours: a tool for cell-based drug testing.
IEEE Trans. Med. Imaging 21, 1212–1221.
(doi:10.1109/TMI.2002.806292)
Debeir O, Van Ham P, Kiss R, Decaestecker C. 2005
Tracking of migrating cells under phase-contrast
video microscopy with combined mean-shift
processes. IEEE Trans. Med. Imaging 24, 697–711.
(doi:10.1109/TMI.2005.846851)
Dufour A, Shinin V, Tajbakhsh S, Guillén-Aghion N,
Olivo-Marin J-C, Zimmer C. 2005 Segmenting and
tracking fluorescent cells in dynamic 3-D
microscopy with coupled active surfaces. IEEE Trans.
Image Process. 14, 1396–1410.
(doi:10.1109/TIP.2005.852790)
Dzyubachyk O, van Cappellen WA, Essers J, Niessen
WJ, Meijering E. 2010 Advanced level-set-based cell
tracking in time-lapse fluorescence microscopy. IEEE
Trans. Med. Imaging 29, 852–867. (doi:10.1109/TMI.
2009.2038693)
Li K, Miller ED, Chen M, Kanade T, Weiss LE,
Campbell PG. 2008 Cell population tracking and
lineage construction with spatiotemporal context.
Med. Image Anal. 12, 546–566. (doi:10.1016/j.media.
2008.06.001)
Maška M, Daněk O, Garasa S, Rouzaut A,
Muñoz-Barrutia A, Ortiz-deSolorzano C. 2013
Segmentation and shape tracking of whole
fluorescent cells based on the Chan–Vese model.
IEEE Trans. Med. Imaging 32, 995–1006.
(doi:10.1109/TMI.2013.2243463)
Padfield D, Rittscher J, Thomas N, Roysam B. 2009
Spatio-temporal cell cycle phase analysis using
level sets and fast marching methods. Med. Image
Anal. 13, 143–155. (doi:10.1016/j.media.2008.06.018)
Deng Y, Coen P, Sun M, Shaevitz JW. 2013 Efficient
multiple object tracking using mutually repulsive
active membranes. PLoS ONE 8, e65769.
(doi:10.1371/journal.pone.0065769)
rsos.royalsocietypublishing.org R. Soc. open sci. 4: 170207
7.
spatio-temporal dynamics. Mol. Microbiol. 80,
612–627. (doi:10.1111/j.1365-2958.2011.
07579.x)
Meijering E, Dzyubachyk O, Smal I. 2012 Methods for
cell and particle tracking. Methods Enzymol. 504,
183–200. (doi:10.1016/B978-0-12-391857-4.
00009-4)
Al-Kofahi O, Radke RJ, Roysam B, Banker G. 2006
Automated semantic analysis of changes in image
sequences of neurons in culture. IEEE Trans. Biomed.
Eng. 53, 1109–1123. (doi:10.1109/TBME.2006.
873565)
Harder N, Mora-Bermúdez F, Godinez WJ, Ellenberg
J, Eils R, Rohr K. 2006 Automated analysis of the
mitotic phases of human cells in 3D fluorescence
microscopy image sequences. In Medical Image
Computing and Computer-Assisted
Intervention–MICCAI 2006, pp. 840–848. Berlin,
Germany: Springer.
Padfield D, Rittscher J, Roysam B. 2008
Spatio-temporal cell segmentation and tracking for
automated screening. In 5th IEEE Int. Symp. on
Biomedical Imaging: from Nano to Macro, 2008. ISBI
2008, pp. 376–379. New York, NY: IEEE.
Meijering E. 2012 Cell segmentation: 50 years down
the road [life sciences]. IEEE Signal Process. Mag. 29,
140–145. (doi:10.1109/MSP.2012.2204190)
Sadanandan SK, Baltekin Ö, Magnusson KE,
Boucharin A, Ranefall P, Jaldén J, Elf J, Wählby C.
2016 Segmentation and track-analysis in time-lapse
imaging of bacteria. IEEE J. Selected Top. Signal
Process. 10, 174–184. (doi:10.1109/JSTSP.2015.
2491304)
Obara B, Roberts MAJ, Armitage JP, Grau V. 2013
Bacterial cell identification in differential
interference contrast microscopy images. BMC
Bioinf. 14, 134. (doi:10.1186/1471-210514-134)
Li F, Zhou X, Ma J, Wong ST. 2010 Multiple nuclei
tracking using integer programming for
quantitative cancer cell cycle analysis. IEEE Trans.
Med. Imaging 29, 96–105. (doi:10.1109/TMI.2009.
2027813)
Schmitter D, Wachowicz P, Sage D, Chasapi A,
Xenarios I, Simanis V, Unser M. 2013 A 2D/3D image
analysis system to track fluorescently labelled
structures in rod-shaped cells: application to
measure spindle pole asymmetry during mitosis.
Cell Div. 8, 6. (doi:10.1186/1747-1028-8-6)
Yilmaz A, Javed O, Shah M. 2006 Object tracking: a
survey. ACM Comput. Surv. 38, 13. (doi:10.1145/
1177352.1177355)
Hue C, LeCadre J-P, Pérez P. 2002 Sequential Monte
Carlo methods for multiple target tracking and data
fusion. IEEE Trans. Signal Process. 50, 309–325.
(doi:10.1109/78.978386)
Smal I, Niessen W, Meijering E. 2008 A new
detection scheme for multiple object tracking in
fluorescence microscopy by joint probabilistic data
association filtering. In 5th IEEE Int. Symp. on
Biomedical Imaging: from Nano to Macro, 2008. ISBI
2008, pp. 264–267. New York, NY: IEEE.
Schwoegler SJ, Blackman SS, Norman RB, Carroll
DE, Capparelli SA. 2014 Multiple hypothesis
tracking. 5 August, US Patent 8,799,189.
Chenouard N, Bloch I, Olivo-Marin J-C. 2013 Multiple
hypothesis tracking for cluttered biological image