Interacting phenotypes and the coevolutionary process: Interspecific indirect genetic effects alter coevolutionary dynamics

Coevolution occurs when species interact to influence one another's fitness, resulting in reciprocal evolutionary change. In many coevolving lineages, trait expression in one species is modified by the genotypes and phenotypes of the other, forming feedback loops reminiscent of models of intraspecific social evolution. Here, we adapt the theory of within-species social evolution, characterized by indirect genetic effects and social selection imposed by interacting individuals, to the case of interspecific interactions. In a trait-based model, we derive general expressions for multivariate evolutionary change in two species and the expected between-species covariance in evolutionary change when selection varies across space. We show that reciprocal interspecific indirect genetic effects can dominate the coevolutionary process and drive patterns of correlated evolution beyond what is expected from direct selection alone. In extreme cases, interspecific indirect genetic effects can lead to coevolution when selection does not covary between species or even when one species lacks genetic variance. Moreover, our model indicates that interspecific indirect genetic effects may interact in complex ways with cross-species selection to determine the course of coevolution. Importantly, our model makes empirically testable predictions for how different forms of reciprocal interactions contribute to the coevolutionary process.