We investigate the effect of counter-rotating-wave terms on the non-Markovianity in quantum open systems by employing the hierarchical equations of motion in the framework of the non-Markovian quantum state diffusion approach. As illustrative examples, the non-Markovian memory effect of a qubit embedded in a bosonic and a fermionic environment at zero temperature are analyzed. It is found that the counter-rotating-wave terms are able to enhance the observed non-Markovianity no matter the environment is composed of bosons or fermions. This result suggests that the rotating-wave approximation may inevitably reduce the non-Markovianity in quantum open systems. Moreover, we find that the modification of the non-Markovianity due to the different statistical properties of environmental modes becomes larger with the increase of the system-environment coupling strength.