The two-point function of exactly marginal operators leads to a universal contribution to the trace anomaly in even dimensions. We study aspects of this trace anomaly, emphasizing its interpretation as a sigma model, whose target space M is the space of conformal field theories (a.k.a. the conformal manifold). When the underlying quantum field theory is supersymmetric, this sigma model has to be appropriately supersymmetrized. As examples, we consider in some detail N=(2,2) and N=(0,2) supersymmetric theories in d=2 and N=2 supersymmetric theories in d=4. This reasoning leads to new information about the conformal manifolds of these theories, for example, we show that the manifold is Kahler-Hodge and we further argue that it has vanishing Kahler class. For N=(2,2) theories in d=2 and N=2 theories in d=4 we also show that the relation between the sphere partition function and the Kahler potential of M follows immediately from the appropriate sigma models that we construct. Along the way we find several examples of potential trace anomalies that obey the Wess-Zumino consistency conditions, but can be ruled out by a more detailed analysis.