We study covariant derivatives on a class of centered bimodules $mathcal{E}$ over an algebra A. We begin by identifying a $mathbb{Z} ( A ) $-submodule $ mathcal{X} ( A ) $ which can be viewed as the analogue of vector fields in this context; $ mathcal{X} ( A ) $ is proven to be a Lie algebra. Connections on $mathcal{E}$ are in one to one correspondence with covariant derivatives on $ mathcal{X} ( A ). $ We recover the classical formulas of torsion and metric compatibility of a connection in the covariant derivative form. As a result, a Koszul formula for the Levi-Civita connection is also derived.