We derive the evolution equation for the second order curvature perturbation using standard techniques of cosmological perturbation theory. We do this for different definitions of the gauge invariant curvature perturbation, arising from different splits of the spatial metric, and compare the expressions. The results are valid at all scales and include all contributions from scalar, vector and tensor perturbations, as well as anisotropic stress, with all our results written purely in terms of gauge invariant quantities. Taking the large-scale approximation, we find that a conserved quantity exists only if, in addition to the non-adiabatic pressure, the transverse traceless part of the anisotropic stress tensor is also negligible. We also find that the version of the gauge invariant curvature perturbation which is exactly conserved is the one defined with the determinant of the spatial part of the inverse metric.