The long-time average behaviour of the value function in the calculus of variations, where both the Lagrangian and Hamiltonian are Tonelli, is known to be connected to the existence of the limit of the corresponding Abel means as the discount factor goes to zero. Still in the Tonelli case, such a limit is in turn related to the existence of solutions of the critical (or, ergodic) Hamilton-Jacobi equation. The goal of this paper is to address similar issues when the Hamiltonian fails to be Tonelli: in particular, for control systems that can be associated with a family of vector fields which satisfies the Lie Algebra rank condition. First, following a dynamical approach we characterise the unique constant for which the ergodic equation admits solutions. Then, we construct a critical solution which coincides with its Lax-Oleinik evolution.