We report the discovery of diffuse extended Ly-alpha emission from redshift 3.1 to 4.5, tracing cosmic web filaments on scales of 2.5-4 comoving Mpc. These structures have been observed in overdensities of Ly-alpha emitters in the MUSE Extremely Deep Field, a 140 hour deep MUSE observation located in the Hubble Ultra Deep Field. Among the 22 overdense regions identified, 5 are likely to harbor very extended Ly-alpha emission at high significance with an average surface brightness of $mathrm{5 times 10^{-20} erg s^{-1} cm^{-2} arcsec^{-2}}$. Remarkably, 70% of the total Ly-alpha luminosity from these filaments comes from beyond the circumgalactic medium of any identified Ly-alpha emitters. Fluorescent Ly-alpha emission powered by the cosmic UV background can only account for less than 34% of this emission at z$approx$3 and for not more than 10% at higher redshift. We find that the bulk of this diffuse emission can be reproduced by the unresolved Ly-alpha emission of a large population of ultra low luminosity Ly-alpha emitters ($mathrm{<10^{40} erg s^{-1}}$), provided that the faint end of the Ly-alpha luminosity function is steep ($alpha lessapprox -1.8$), it extends down to luminosities lower than $mathrm{10^{38} - 10^{37} erg s^{-1}}$ and the clustering of these Ly-alpha emitters is significant (filling factor $< 1/6$). If these Ly-alpha emitters are powered by star formation, then this implies their luminosity function needs to extend down to star formation rates $mathrm{< 10^{-4} M_odot yr^{-1}}$. These observations provide the first detection of the cosmic web in Ly-alpha emission in typical filamentary environments and the first observational clue for the existence of a large population of ultra low luminosity Ly-alpha emitters at high redshift.