Using the recently constructed covariant Ito-Langevin dynamics, we develop a covariant theory of non-equilibrium thermodynamics that is applicable to small systems with multiplicative noises and with slow variables forming curved manifolds. Assuming instantaneous detailed balance, we derive expressions for work, heat, entropy production, and free energy both at ensemble level, as well as at the level of individual dynamic trajectory. We also relate time-reversal asymmetry to entropy production, and derive its consequences such as fluctuation theorem and work relation. The theory is based on Ito-calculus, is fully covariant under time-independent nonlinear transformation of variables, and is applicable to systems strongly coupled to environments.