Model systems of self-propelled particles reproduce many phenomena observed in laboratory active matter systems that defy our thermal equilibrium-based intuition. In particular, in stationary states of self-propelled systems, it is recognized that velocities of different particles exhibit non-trivial equal-time correlations. Such correlations are absent in equivalent equilibrium systems. Recently, researchers found that the range of the velocity correlations increases with increasing persistence time of the self-propulsion and can extend over many particle diameters. Here we review the initial studies of long-ranged velocity correlations in solid-like systems of self-propelled particles. Then, we demonstrate that the long-ranged velocity correlations are also present in dense fluid-like systems. We show that the range of velocity correlations in dense systems of self-propelled particles is determined by the combination of the self-propulsion and the virial bulk modulus that originates from repulsive interparticle interactions.