The large values of the singlet and triplet two-nucleon scattering lengths locate the nuclear system close to the unitary limit. This particular position strongly constrains the low-energy observables in the three-nucleon system as depending on one parameter, the triton binding energy, and introduces correlations in the low energy sector of light nuclei. Here we analyze the propagation of these correlations to infinite nuclear matter showing that its saturation properties, the equation of state of $beta$-stable nuclear matter and several properties of neutron stars, as their maximum mass, are well determined solely by a few number of low-energy quantities of the two- and three-nucleon systems. In this way we make a direct link between the universal behavior observed in the low-energy region of few-nucleon systems and fundamental properties of nuclear matter and neutron stars.