The orbits of the least chemically enriched stars open a window on the formation of our Galaxy when it was still in its infancy. The common picture is that these low-metallicity stars are distributed as an isotropic, pressure-supported component since these stars were either accreted from the early building blocks of the assembling Milky Way, or were later brought by the accretion of faint dwarf galaxies. Combining the metallicities and radial velocities from the Pristine and LAMOST surveys and Gaia DR2 parallaxes and proper motions for an unprecedented large and unbiased sample of very metal-poor stars at $[Fe/H]leq-2.5$ we show that this picture is incomplete. This sample shows strong statistical evidence (at the $5.0sigma$ level) of asymmetry in their kinematics, favouring prograde motion. Moreover, we find that $31%$ of the stars that currently reside in the disk do not venture outside of the disk plane throughout their orbit. The discovery of this population implies that a significant fraction of stars with iron abundances $[Fe/H]leq-2.5$ formed within or concurrently with the Milky Way disk and that the history of the disk was quiet enough to allow them to retain their disk-like orbital properties.