We present a comprehensive study of the low-energy band structure and Fermi surface (FS) topology of $A$Co$_2$As$_2$ ($A=$ Ca, Sr, Ba, Eu) using high-resolution angle-resolved photoemission spectroscopy. The experimental FS topology and band dispersion data are compared with theoretical full-potential linearized augmented-plane-wave (FP-LAPW) calculations, which yielded reasonably good agreement. We demonstrate that the FS maps of $A$Co$_2$As$_2$ are significantly different from those of the parent compounds of Fe-based high-temperature superconductors. Further, the FSs of CaCo$_2$As$_2$ do not show significant changes across its antiferromagnetic transition temperature. The band dispersions extracted in different momentum $(k_{it x}, k_{it y})$ directions show a small electron pocket at the center and a large electron pocket at the corner of the Brillouin zone (BZ). The absence of the hole FS in these compounds does not allow nesting between pockets at the Fermi energy ({it E}$_{rm F}$), which is in contrast to $A$Fe$_2$As$_2$-type parent compounds of the iron-based superconductors. Interestingly, we find that the hole bands are moved 300--400~meV below $E_{rm F}$ depending on the $A$ element. Moreover, the existence of nearly flat bands in the vicinity of $E_{rm F}$ are consistent with the large density of states at $E_{rm F}$. These results are important to understand the physical properties as well as the possibility of the emergence of superconductivity in related materials.