We study the molecular gas content of 24 star-forming galaxies at $z=3-4$, with a median stellar mass of $10^{9.1}$ M$_{odot}$, from the MUSE Hubble Ultra Deep Field (HUDF) Survey. Selected by their Lyman-alpha-emission and H-band magnitude, the galaxies show an average EW $approx 20$ angstrom, below the typical selection threshold for Lyman Alpha Emitters (EW $> 25$ angstrom), and a rest-frame UV spectrum similar to Lyman Break Galaxies. We use rest-frame optical spectroscopy from KMOS and MOSFIRE, and the UV features observed with MUSE, to determine the systemic redshifts, which are offset from Lyman alpha by 346 km s$^{-1}$, with a 100 to 600 km s$^{-1}$ range. Stacking CO(4-3) and [CI](1-0) (and higher-$J$ CO lines) from the ALMA Spectroscopic Survey of the HUDF (ASPECS), we determine $3sigma$ upper limits on the line luminosities of $4.0times10^{8}$ K km s$^{-1}$pc$^{2}$ and $5.6times10^{8}$ K km s$^{-1}$pc$^{2}$, respectively (for a 300 km s$^{-1}$ linewidth). Stacking the 1.2 mm and 3 mm dust continuum flux densities, we find a $3sigma$ upper limits of 9 $mu$Jy and $1.2$ $mu$Jy, respectively. The inferred gas fractions, under the assumption of a Galactic CO-to-H$_{2}$ conversion factor and gas-to-dust ratio, are in tension with previously determined scaling relations. This implies a substantially higher $alpha_{rm CO} ge 10$ and $delta_{rm GDR} ge 1200$, consistent with the sub-solar metallicity estimated for these galaxies ($12 + log(O/H) approx 7.8 pm 0.2$). The low metallicity of $z ge 3$ star-forming galaxies may thus make it very challenging to unveil their cold gas through CO or dust emission, warranting further exploration of alternative tracers, such as [CII].