We use gauge-invariant cosmological perturbation theory to calculate the displacement field that sets the initial conditions for $N$-body simulations. Using first and second-order fully relativistic perturbation theory in the synchronous-comoving gauge, allows us to go beyond the Newtonian predictions and to calculate relativistic corrections to it. We use an Einstein--de Sitter model, including both growing and decaying modes in our solutions. The impact of our results should be assessed through the implementation of the featured displacement in cosmological $N$-body simulations.