Beerli, P. and J. Felsenstein (2001). “Maximum likelihood estimation of a migration matrix and effective population sizes in n subpopulations by using a coalescent approach.” Proceedings of the National Academy of Sciences of the United States of America 98(8): 4563-4568.
Beerli and Felsenstein (Beerli and Felsenstein, 1999) developed maximum-likelihood and coalescent theory based method to estimate migration rate and effective population size. The likelihood parameter Θ= [Θ1, Θ2, М1, М2], where Θ is 4Neμ and Μ is m/μ, are calculated by exploring genealogical trees, including topology of branch length and migration scenarios. Rather than exploring all possible genealogical trees, parameters are calculated focusing the trees with high likelihood using Markov chain Monte Carlo approach. After each chain, parameters are recalculated and the likelihood is reevaluated. To obtain more accurate estimate, this process is repeated for many times.
In 2001, Beerli and Felsenstein (Beerli and Felsenstein, 2001) developed a program called MIGRATE. MIGRATE use n-Island model (n is number of subpopulations) and estimates asymmetrical migration rates (Μ) between demes and effective population size of demes (Θ) for more than three sampled populations at same time. Multiplying Μ and Θ gives Nem. Because gene flow influence within-population genetic diversity and MIGRATE estimates Θ accounting for gene flow (Ray et al., 2003), MIGRATE should provide more accurate estimates of effective population size than traditional methods (See also here).
Unfortunately, human populations usually violate the important assumptions to estimate parameters, so the interpretation of the data has to be treated with cautions. First, MIGRATE assumes that there is no unsampled population exchanging genes with sampled populations. Beerli (Beerli, 2004) examined the effects of unsampled populations on these methods. He found that migration rate, Μ=m/μ, is not seriously affected, but effective population size is upwardly biased and Nem tends to be overestimated. Beerli suggests that more accurate estimation can be obtained by running the program with many populations at the same time. However, human populations tend to interact with many different populations and many of them are not sampled.
Second, MIGRATE assumes that population size and migration rate did not change over time. However, many human populations experienced demographic expansion or bottleneck in the past. Development of better transportation technologies, recent state expansion, European contact, globalization, and industrialization increased movement of people over time.