Ca$_3$Ti$_2$O$_7$ is an experimentally confirmed hybrid improper ferroelectric material, in which the electric polarization is induced by a combination of the coherent TiO$_6$ octahedral rotation and tilting. In this work, we investigate the tuning of ferroelectricity of Ca$_3$Ti$_2$O$_7$ using iso-valent substitutions on Ca-sites. Due to the size mismatch, larger/smaller alkaline earths prefer $A$/$A$ sites respectively, allowing the possibility for site-selective substitutions. Without extra carriers, such site-selected iso-valent substitutions can significantly tune the TiO$_6$ octahedral rotation and tilting, and thus change the structure and polarization. Using the first-principles calculations, our study reveals that three substituted cases (Sr, Mg, Sr+Mg) show divergent physical behaviors. In particular, (CaTiO$_3$)$_2$SrO becomes non-polar, which can reasonably explain the suppression of polarization upon Sr substitution observed in experiment. In contrast, the polarization in (MgTiO$_3$)$_2$CaO is almost doubled upon substitutions, while the estimated coercivity for ferroelectric switching does not change. The (MgTiO$_3$)$_2$SrO remains polar but its structural space group changes, with moderate increased polarization and possible different ferroelectric switching paths. Our study reveals the subtle ferroelectricity in the $A_3$Ti$_2$O$_7$ family and suggests one more practical route to tune hybrid improper ferroelectricity, in addition to the strain effect.