Abstract: The nested Kingman Coalescent $(K^n_t, t\geq 0)$ is a model for the genealogy of lineages belonging to different species, which can be thought of as (gene-)trees within a (species-)tree. We assume that the species tree is given as a Kingman tree and once species coalesce the inner gene lineages are allowed to coalesce as well.
Here, we focus on the empirical measure $g^n_{t}= \frac{1}{s^n_t} \sum_{i=1}^{s_t^n} \delta_{\Pi_t^n (i)}$ of block sizes in the nested coalescent, where $\Pi_t^n (i)$ denotes the number of gene lineages in species $i$. As the number of species tends to infinity we show convergence towards a solution $u(t,x)$ of the Smoluchowski coagulation-transport equation.
In particular, if there are fewer lineages per species than species itself, $g_t^n$ converges to a dust solution $u(t,x)$, meaning that $u(t,x) \to \delta_0 (dx)$ as $t\to 0$. Dust solutions have appeared in [Lambert and Schertzer 2020] and we show that dust solutions can be expressed in terms of the Brownian Snake. Through that connection we prove existence and uniqueness (up to scaling) for solutions of this equation.