If two species exhibit different nonlinear responses to a single shared resource, and if each species modifies the resource dynamics such that this favors its competitor, they may stably coexist. This coexistence mechanism, known as relative nonlinea
rity of competition, is well understood theoretically, but less is known about its evolutionary properties and its prevalence in real communities. We address this challenge by using adaptive dynamics theory and individual-based simulations to compare community stabilization and evolutionary stability of species that coexist by relative nonlinearity. In our analysis, evolution operates on the species density-compensation strategies, and we consider a trade-off between population growth rates at high and low resource availability. We confirm previous findings that, irrespective of the particular model of density dependence, there are many combinations of overcompensating and undercompensating density-compensation strategies that allow stable coexistence by relative nonlinearity. However, our analysis also shows that most of these strategy combinations are not evolutionarily stable and will be outcompeted by an intermediate density-compensation strategy. Only very specific trade-offs lead to evolutionarily stable coexistence by relative nonlinearity. As we find no reason why these particular trade-offs should be common in nature, we conclude that the sympatric evolution and evolutionary stability of relative nonlinearity, while possible in principle, seems rather unlikely. We speculate that this may, at least in part, explain why empirical demonstrations of this coexistence mechanism are rare, noting, however, that the difficulty to detect relative nonlinearity in the field [...]
