The nature of quantum spin liquids is studied for the spin-$1/2$ antiferromagnetic Heisenberg model on a square lattice containing exchange interactions between nearest-neighbor sites, $J_1$, and those between next-nearest-neighbor sites, $J_2$. We perform variational Monte Carlo simulations together with the quantum-number-projection technique and clarify the phase diagram in the ground state together with its excitation spectra. We obtain the nonmagnetic phase in the region $0.4< J_2/J_1le 0.6$ sandwiched by the staggered and stripe antiferromagnetic (AF) phases. Our direct calculations of the spin gap support the notion that the triplet excitation from the singlet ground state is gapless in the range of $0.4 < J_2/J_1 le 0.5$, while the gapped valence-bond-crystal (VBC) phase is stabilized for $0.5 < J_2/J_1 le 0.6$. The VBC order is likely to have the columnar symmetry with a spontaneous symmetry breaking of the $C_{4v}$ symmetry. The power-law behaviors of the spin-spin and dimer-dimer correlation functions in the gapless region are consistent with the emergence of the algebraic quantum-spin-liquid phase (critical phase). The exponent of the spin correlation $langle S(0)S(r)rangle propto 1/r^{z+eta}$ at a long distance $r$ appears to increase from $z+etasim 1$ at $J_2/J_1sim0.4$ toward the continuous transition to the VBC phase at $J_1/J_1sim0.5$. Our results, however, do not fully exclude the possibility of a direct quantum transition between the staggered AF and VBC phases with a wide critical region and deconfined criticality.