We give examples of infinite order rational transformations that leave linear differential equations covariant. These examples are non-trivial yet simple enough illustrations of exact representations of the renormalization group. We first illustrate covariance properties on order-two linear differential operators associated with identities relating the same $_2F_1$ hypergeometric function with different rational pullbacks. We provide two new and more general results of the previous covariance by rational functions: a new Heun function example and a higher genus $_2F_1$ hypergeometric function example. We then focus on identities relating the same hypergeometric function with two different algebraic pullback transformations: such remarkable identities correspond to modular forms, the algebraic transformations being solution of another differentially algebraic Schwarzian equation that emerged in a paper by Casale. Further, we show that the first differentially algebraic equation can be seen as a subcase of the last Schwarzian differential condition, the restriction corresponding to a factorization condition of some associated order-two linear differential operator. Finally, we also explore generalizations of these results, for instance, to $_3F_2$, hypergeometric functions, and show that one just reduces to the previous $_2F_1$ cases through a Clausen identity. In a $_2F_1$ hypergeometric framework the Schwarzian condition encapsulates all the modular forms and modular equations of the theory of elliptic curves, but these two conditions are actually richer than elliptic curves or $_2F_1$ hypergeometric functions, as can be seen on the Heun and higher genus example. This work is a strong incentive to develop more differentially algebraic symmetry analysis in physics.