Using results of the band structure calculations in the local-spin-density approximation we demonstrate how the crystal distortions affect the magnetic structure of orthorhombically distorted perovskites leading to a non-collinear spin arrangement. Our results suggest that the non-collinearity of the spin magnetic moments, being generally small in La$M$O$_3$ series with $M$=Cr-Fe, is large in SrRuO$_3$.