Abstract

The transfer of genetic material between species, commonly known as interspecific gene flow, is an important determinant of population genetic variation in many species. Introgressed fragments along the genome contain information about the timing and extent of contact between the ancestors of present-day species. However, gene flow can be challenging to characterize accurately because one of its key signatures—gene trees that are topologically discordant from a species tree—also happens in the presence of incomplete lineage sorting (ILS). This phenomenon, proven to be pervasive across taxa, arises when different genetic lineages within a species do not sort out completely before speciation occurs, generating instances of deep coalescence where genealogies might have a topology which is discordant with the species-level tree. In an attempt to model ILS, we previously developed TRAILS, a hidden Markov model that explicitly uses the information contained in ILS fragments to obtain unbiased estimates of effective population sizes and speciation times. Here, I present TRAILS v2, an extension of the original TRAILS framework that adds new functionality by using an expanded hidden state space to accommodate pulse-like unidirectional introgression within the coalescent with recombination. By jointly modeling ILS and interspecific gene flow, TRAILS v2 can accurately infer the timing and ancestral contributions of hybridization events, which we validate with extensive coalescent simulations. Additionally, the posterior decoding of TRAILS v2 can be used to discriminate between gene flow and ILS at the base-pair level, and to infer coalescent times of different genomic fragments in discretized time intervals, thus reconstructing the multi-species ancestral recombination graph. Because ILS and interspecific gene flow are two very common processes, TRAILS v2 can potentially be applied to many different study systems. For example, TRAILS v2 can be used to infer the extent and timing of Neanderthal gene flow into the ancestors of modern non-African populations, together with timing the out-of-Africa migration event, estimating ancestral effective population sizes in archaic humans, and distinguishing ILS fragments from true cases of Neanderthal introgression.