The widely used crystal structures for both heptazine-based and triazine-based two-dimensional (2D) graphitic carbon nitride (g-C$_3$N$_4$) are the flat P-6m2 configurations. However, the experimentally synthesized 2D g-C$_3$N$_4$ possess thickness ranging in 0.2-0.5 nm, indicating that the theoretically used flat P-6m2 configurations are not the correct ground states. In this work, we propose three new corrugated structures P321, P3m1 and Pca21 with energies of 66 (86), 77 (87) and 78 (89) meV/atom lower than that of the corresponding heptazine-based (triazine-based) g-C$_3$N$_4$ in flat P-6m2 configuration, respectively. These corrugated structures have very similar periodic patterns to the flat P-6m2 ones and they are difficult to be distinguished from each other according to their top-views. The optimized thicknesses of the three corrugated structures ranging in 1.347-3.142 {AA} are in good agreement with the experimental results. The first-principles results show that these corrugated structural candidates are also semiconductors with band gaps slightly larger than those of the correspondingly flat P-6m2 ones. Furthermore, they possess also suitable band edge positions for sun-light-driven water-splitting at both $pH=0$ and $pH=7$ environments. Our results show that these three new structures are more promising candidates for the experimentally synthesized g-C$_3$N$_4$.