In this paper we study Higgs and co-Higgs $G$-bundles on compact Kahler manifolds $X$. Our main results are: (1) If $X$ is Calabi-Yau, and $(E,,theta)$ is a semistable Higgs or co-Higgs $G$-bundle on $X$, then the principal $G$-bundle $E$ is semistable. In particular, there is a deformation retract of ${mathcal M}_H(G)$ onto $mathcal M(G)$, where $mathcal M(G)$ is the moduli space of semistable principal $G$-bundles with vanishing rational Chern classes on $X$, and analogously, ${mathcal M}_H(G)$ is the moduli space of semistable principal Higgs $G$-bundles with vanishing rational Chern classes. (2) Calabi-Yau manifolds are characterized as those compact Kahler manifolds whose tangent bundle is semistable for every Kahler class, and have the following property: if $(E,,theta)$ is a semistable Higgs or co-Higgs vector bundle, then $E$ is semistable.