Journal of Heredity, 2015, 745–748
doi:10.1093/jhered/esv082
Brief Communication
Advance Access publication October 12, 2015
Brief Communication
Estimation of Male Gene Flow: Use Caution
Philip W. Hedrick, Sujeet Singh, and Jouni Aspi
From the School of Life Sciences, Arizona State University, Tempe, AZ 85287 (Hedrick); Department of Genetics and
Physiology, University of Oulu, Oulu, Finland (Singh and Aspi); and Wildlife Institute of India, Dehradun, Uttarakhand,
India (Singh).
Address correspondence to Philip W. Hedrick at the address above, or e-mail: [email protected].
Received April 19, 2015; First decision June 21, 2015; Accepted September 23, 2015.
Corresponding Editor: Fred Allendorf
Abstract
Because male gene flow cannot easily be estimated directly in many organisms, Hedrick et al.
(2013) provided an approach to estimate male gene flow given estimates of diploid nuclear and
female differentiation. This approach appears to work well when there is lower female than male
gene flow. However, in a tiger data set there was less female differentiation observed as estimated
by mitochondrial DNA than expected given the observed overall nuclear diploid differentiation. To
analyze these data, we suggest an alternative approach which allows incorporation of sex-specific
gene flow and sex-specific effective population size. We find that the pattern of differentiation
observed in tigers was consistent with a lower male than female effective population size using
this alternative approach. Further, this finding is consistent with observed data in tigers where the
male effective population size was 33% that of the female effective population size.
Subject areas: Population structure and phylogeography; Conservation genetics and biodiversity
Key words: effective population size, gene flow, mtDNA, tigers, Y chromosomes
The amount of genetic differentiation from male gene flow can be
directly estimated if there are paternally inherited, haploid Y chromosomes. However, in many organisms there are either no Y chromosomes or no Y chromosome markers known, so Hedrick et al.
(2013) proposed an approach to estimate male gene flow when there
are estimates of diploid nuclear and female differentiation. That is,
given the differentiation for diploid nuclear genes, FST, and the differentiation for females from mitochondrial DNA (mtDNA), FST(f),
they gave the estimated differentiation for males as
FST (m) =
2FST FST (f )
FST (f ) − 2FST + 3FST FST (f )
(1a)
and the ratio of male gene flow mm to female gene flow mf as
mm FST (f ) (1 − FST (m) )
=
FST (m) (1 − FST (f ) )
mf
(these were equations 7a and 7b in Hedrick et al. 2013).
(1b)
Singh et al. (2015) estimated the level of diploid nuclear differentiation (using 9 microsatellite loci) and mtDNA differentiation in 3 Indian populations of Bengal tigers and applied the
approach above. Table 1 gives their estimated values of FST and
FST(f) between the 3 population pairs and the estimates of and
mm/mf and FST(m) using the expressions above (indicated by an *
in Table 1). For the population pair Peninsular-Sundarbans, the
estimate of FST(m) was greater than unity, the upper bound for this
measure, and the estimate of the ratio mm/mf was less than zero,
the lower bound for this ratio. For the population pair NorthernPeninsular, the estimate of FST(m) was at the upper bound for this
measure.
To gain some perspective on the relationship of these differentiation values, let us first assume that FST(f) and FST(m) are equal, or
FST(f = m). In this case, Equation (1a) can be solved for FST as
FST =
© The American Genetic Association. 2015. All rights reserved. For permissions, please e-mail: [email protected]
FST (f = m)
4 − 3FST (f = m)
(2)
745
Journal of Heredity, 2015, Vol. 106, No. 6
746
Table 1. The observed overall and female differentiation in 3 comparisons of Bengal tigers populations and that with equal sex-specific
gene flow, higher male than female gene flow, equal effective population size in the 2 sexes, and lower effective population size in males
that give equilibrium differentiation values for nuclear and female markers from Equation (4) close to that observed
Northern-Peninsular
Northern-Sundarbans
Peninsular-Sundarbans
Observed
mm = mf
mm = 2mf
Nm = Nf
Nm = Nf/2
Observed
mm = mf
mm = 2mf
Nm = Nf
Nm = Nf/2
Observed
mm = mf
mm = 2mf
Nm = Nf
Nm = Nf/2
mm
mf
mm/mf
Nm
Nf
Nm/Nf
FST
FST(f)
FST(m)
—
0.05
0.1
0.07
0.1
—
0.05
0.1
0.08
0.08
—
0.05
0.1
0.01
0.03
—
0.05
0.05
0.25
0.15
—
0.05
0.05
0.07
0.035
—
0.05
0.05
0.15
0.085
−0.01*
1
2
0.28
0.67
1.21*
1
2
1.14
2.29
−0.11*
1
2
0.067
0.35
—
45
30
20
20
—
38
15
20
20
—
20
12
20
20
—
150
150
20
40
—
27
27
20
40
—
70
70
20
40
—
0.3
0.2
1
0.5
—
1.41
0.56
1
0.5
—
0.29
0.17
1
0.5
0.030
0.032
0.029
0.033
0.030
0.070
0.068
0.071
0.069
0.069
0.069
0.069
0.067
0.064
0.069
0.058
0.058
0.058
0.057
0.061
0.250
0.255
0.255
0.242
0.253
0.116
0.117
0.117
0.115
0.114
1.081*
0.171
0.124
0.176
0.176
0.215*
0.204
0.221
0.216
0.216
7.956*
0.316
0.262
0.711
0.443
*Estimates from Equations (1a) and (1b).
FST (f ) =
2FST FST (m)
FST (m) − 2FST + 3FST FST (m)
(3a)
Then if FST(m) = 1, the maximum value it can have when there is no
male gene flow, then
FST (f ) >
2FST
1 + FST
(3b)
Or, given the maximum male differentiation possible, FST(f) must be
approximately greater than twice FST. For example, if FST = 0.069,
as in the Peninsular-Sundarbans comparison, then FST(f) must be
1
0.8
FST(m)
In other words, if FST(f = m) is small, then FST is approximately FST(f = m)/4,
a result not unexpected because the effective population size for
diploid nuclear markers is approximately 4 times that for haploid
markers inherited in only one sex when the effective population size
is similar in males and females. In the examples of American bison
and California sea lions given previously by Hedrick et al. (2013),
the values of FST were less than FST(f)/4 and the values of FST(m) were
within the appropriate bounds.
Using Equation (2), it is useful to point out explicitly the difference in the expected size of differentiation for diploid, nuclear
markers and haploid, sex-specific markers. When gene flow is equal
for the 2 sexes, the expected value of differentiation measured by
FST is much less than that for either sex measured by FST(f) or FST(m)
because of the lower effective population sizes for the mtDNA and
Y markers used to measure these differentiation values. It does not
mean, as sometimes suggested, that if FST < FST(f) that there is strong
sex-biased gene flow, more male than female gene flow. This conclusion could only be potentially made if FST < FST(f)/(4 − 3 FST(f)) as in
the American bison and California sea lion examples discussed by
Hedrick et al. (2013).
Now let us examine the bounds on the range of FST(f) for a given
FST value assuming that FST(m) cannot be greater than unity, the
maximum value when there is no male gene flow. We recognize that
estimates of differentiation values do not necessarily have a lower
bound of 0 but that they have an upper bound of 1. To do this, we
can rearrange Equation (1a) for FST(f) as
0.6
0.4
FST = 0.10
0.2
FST = 0.05
0
0
0.2
0.4
0.6
0.8
1
FST(f)
Figure 1. The value of FST(m) calculated from equation (1a) for FST = 0.05 or
0.1 and different values of FST(f) where the solid circles indicate values when
FST(f) = FST(m).
larger than 0.129. The FST(f) estimate from Singh et al. (2015) was
only 0.116, lower than this bound. This demonstrates that using the
approach above to estimate FST(m) can give values greater than one if
FST(f) is low relative to FST.
Figure 1 gives the estimated FST(m) values from Equation (1a),
given that FST was equal to 0.05 or 0.1. The lower bound of FST(f)
when the FST(m) = 1 is 0.095 for FST = 0.05 and 0.18 for FST = 0.1. The
value when FST(f) = FST(m) is given by the solid circles in Figure 1, 0.174
and 0.308 for FST = 0.05 and 0.1, respectively. When FST(f) is greater
than this value, FST(m) is defined and becomes small (high male gene
flow). On the other hand when FST(f) is below the equal value in the 2
sexes, FST(m) quickly increases until it reaches its maximum FST(m) = 1
when there is no male gene flow.
As discussed in Hedrick et al. (2013), the estimates of FST(m) and
mm/mf they derived were based on several assumptions, such as the
populations were at gene flow-genetic drift equilibrium, they were
not significantly influenced by some other factors, such as different
Journal of Heredity, 2015, Vol. 106, No. 6
747
male and female effective population sizes, and they were based on
some approximations. In addition in the derivation for Equation
(1a), the female and mean gene flow at equilibrium were assumed to
be a function of the male and overall effective population sizes, not
independent of them, and result in the constraints discussed above.
Some of these assumptions appear to have resulted in the restricted
possible range of values discussed above and the extreme values
found in consequent estimates for the Bengal tiger data.
We have assumed in the discussion above that the values of FST
and FST(f) are estimated without error. We recognize that a larger sample size should improve the estimation of these statistics and that a
larger number of independent loci should improve the estimation
of FST. However, because the mitochondrial genes are haploid, the
sample size is inherently smaller than for diploid genes. Also because
all mitochondrial genes are inherited as a unit (Avise 2000), data
from more mitochondrial genes might increase the resolution of
the female differentiation estimate but more mitochondrial genes
would likely not provide independent genetic information to greatly
improve the estimation of FST(f).
As a possible alternative to the above approach, we suggest the
approach below. This approach appears particularly useful when
FST(f) is low relative to FST as we saw in data for the PeninsularSundarbans population pair. Hedrick et al. (2013) gave the following
approach to evaluate the overall differentiation and differentiation
in the 2 sexes where the effects of sex-specific gene flow and effective population sizes are included. That is, we can determine sexspecific and overall differentiation by using the following 3 recursion
equations
 1
1  t 

2
t +1
FST
=
+ 1 −
 FST  (1 − m) 2N 
 2N 

 1 
1  t 
t +1
2
FST
+ 1 −
(f ) = 
 FST (f )  (1 − mf ) 
N
N
f
f


 1

1  t 
t +1
FST
+ 1 −
FST (m)  (1 − mm )2
(m) = 
N m 
 Nm 

(4)
(Wright 1951). These expressions can be used to determine the
change in genetic differentiation values over time from generation
t to t + 1 and their eventual equilibrium values, where N, Nf, and
Nm are the total, female, and male effective population sizes, respectively, and m is the mean gene flow.
Using these expressions, there are more independent parameters than in the approach used by Hedrick et al. (2013) to derive
Equation (1a). For example, Nf and Nm are independent of each
other [N = 4NmNf /(Nm + Nf)], mm and mf are independent of each
other [m = (mm + mf)/2] and of Nf and Nm. On the other hand, in the
derivation of Equation (1a), it was assumed that the FST values were
a function of the product of the N and m values.
Using the expressions in Equation (4), and iterating them until
they are at equilibrium (generally 10–50 generations depending
upon the parameters and starting FST values), the combinations given
in Table 1 are consistent with the FST and FST(f) values observed in the
comparisons of the three Bengal tiger comparisons. For example,
in the Northern-Peninsular comparison, the observed values were
FST = 0.030 and FST(f) = 0.058 (as given in the first row) and these values can be explained by the male and female gene flow and effective
population size values given in the next 4 rows.
When the sex-specific gene flow levels are set to be the same
(here mm = mf = 0.05), then the observed differentiation levels are
explained by Nm = 45 and Nf = 150. In other words, a male effective population size 30% that of the female effective population size
is consistent with the observed differentiation values, given equivalent gene flow in the 2 sexes. Note also that for this combination of
parameters, male differentiation FST(m) is 0.155, much higher than
that for females.
Second, if the male and female effective population sizes are kept
equal (here Nm = Nf = 20), then the observed differentiation values in
the Northern-Peninsular comparison are consistent with mm = 0.07
and mf = 0.25, a rate of gene flow in males 28% of that in females.
Again, with these gene flow values, differentiation estimated in males
is much higher than that observed for females. Notice that these 2
different explanations give male to female gene flow ratios mm/mf
equal to 1 and 0.28, respectively.
Other combinations of sex-specific effective populations and
sex-specific gene flow values can also give values of differentiation
consistent with the observed differentiation values. As examples,
Table 1 gives examples where there is lower male gene flow than
female gene flow, mm = 2mf and a lower male effective population
size than female effective population size, Nm = Nf/2.
In Bengal tigers, the male effective size appears to be much
lower than the female effective population because the variance
in male reproductive success is much higher than that in females
(Smith and McDougal 1991). Smith and McDougal (1991) found
that there were 20 breeding males and 45 breeding females in
the Chitwan population from Nepal. From lifetime reproduction
data, they estimated the ratio of the effective number of males
to breeding male number as 0.43 and the ratio of the effective
number of females to breeding male number as 0.68. As a result,
the effective number of males Nm was about 9 and the effective
number of females Nf was about 27 for a ratio of Nm/Nf = 0.33,
similar to that used in Table 1 when the gene flow of the 2 sexes
are equal. Khan (2004) also suggested that natural mortality is
higher in males and that 73% of poaching mortality occurs in
males, factors that could contribute to the lower effective population size in males than in females. This independent information suggests that the sets of values in which the male effective
population size is smaller than the female effective population
size (rows 2 and 3 for each of the population pairs in Table 1)
are reasonable explanations for the observed differentiation
patterns.
In addition to these explanations of the observed FST and FST(f)
values, nonequilibrium situations can give similar differentiation
values. Also, as we mentioned above, error in the estimation FST and
FST(f) could also give values potentially like those observed. However,
the general impression from the tiger data is that the value of FST(f)
is low relative to FST, that is, there was less female differentiation
observed as estimated by mtDNA than expected given the overall
differentiation. Because mtDNA is haploid and is inherited as a single unit, the expectation is that if estimation error were important
that it would generally increase the variability of mtDNA allele frequencies over samples from different populations (and potentially
increase FST(f)) more than the variability of nuclear allele frequencies
over samples from different populations, however, here the opposite
appears to be present.
Overall, it is difficult, given nuclear and female differentiation
values, to confidently estimate specific male gene flow levels without
other information. The approach developed by Hedrick et al. (2013)
appears to provide estimates that are consistent with expectations
Journal of Heredity, 2015, Vol. 106, No. 6
748
in situations with lower female than male gene flow but in other
situations this approach can provide estimates that are beyond the
known bounds of the parameters. In such a situation, we have used
an approach iterating expected changes in differentiation that allows
gene flow and effective population size parameters to be independent of each other. We have found that very different parameter sets
can give levels of nuclear and female differentiation similar to that
observed. If other data, such as information about sex-specific gene
flow or sex-specific effective population size are known, it can be
used to suggest which estimates are most reasonable, as we have here
for the Bengal tiger example.
Although data from paternally inherited, Y chromosomes could
provide a direct measure of male differentiation, Y chromosome
markers are often not known. However, like mtDNA they would
generally represent a single, haploid genetic unit and have more estimation error than diploid genes. As a result, the approach discussed
here could provide a framework for examining how evolutionary
parameters result in observed differentiation patterns including
observed male differentiation.
References
Avise JC. 2000. Phylogeography. Cambridge (MA): Harvard University Press.
Hedrick PW, Allendorf FW, Baker CS. 2013. Estimation of male gene flow
from measures of nuclear and female genetic differentiation. J Hered.
104:713–717.
Khan MMH. 2004. Ecology and conservation of the Bengal tiger in the Sundarbans mangrove forest of Bangladesh [PhD thesis]. [Cambridge (UK)]:
University of Cambridge.
Singh SJ, Mishra S, Aspi J, Kvist L, Nigam P, Pandey P, Sharma R, Goyal SP.
2015. Tigers of Sundarbans in India: is the population a separate conservation unit? PLoS One. 10:e0118846.
Smith JLD, McDougal C. 1991. The contribution of variance in lifetime reproduction to effective population size in tigers. Conserv Biol. 5:484–490.
Wright S. 1951. The genetical structure of populations. Ann Eugen. 15:323–
354.