I search for the ground state structures of the kagome metals KV$_3$Sb$_5$, RbV$_3$Sb$_5$, and CsV$_3$Sb$_5$ using first principles calculations. Group-theoretical analysis shows that there are seventeen different distortions that are possible due to the phonon instabilities at the $M$ $(frac{1}{2},0,0)$ and $L$ $(frac{1}{2},0,frac{1}{2})$ points in the Brilouin zone of the parent $P6/mmm$ phase of these materials. I generated these structures for the three compounds and performed full structural relaxations that minimize the atomic forces and lattice stresses. I find that the $Fmmm$ phase with the order parameter $M_1^+$ $(a,0,0)$ $+$ $L_2^-$ $(0,b,b)$ has the lowest energy among these possibilities in all three compounds. However, the $Fmmm$ exhibits a dynamical instability at its $Z$ $(0,0,1)$ point, which corresponds to the $A$ $(0,0,frac{1}{2})$ point in the parent $P6/mmm$ phase. Condensation of this instability leads to a base-centered orthorhombic structure with the space group $Cmcm$ and $4Q$ order parameter $M_1^+$ $(a,0,0)$ $+$ $L_2^-$ $(0,b,b)$ $+$ $A_6^+$ $(frac{1}{2}c,frac{-sqrt{3}}{2}c)$.