With a scalar potential and a bivector potential, the vector field associated with the drift of a diffusion is decomposed into a generalized gradient field, a field perpendicular to the gradient, and a divergence-free field. We give such decomposition a probabilistic interpretation by introducing cycle velocity from a bivectorial formalism of nonequilibrium thermodynamics. New understandings on the mean rates of thermodynamic quantities are presented. Deterministic dynamical system is further proven to admit a generalized gradient form with the emerged potential as the Lyapunov function by the method of random perturbations.