This paper describes precise measurements of the thermal neutron flux in the LSM underground laboratory in proximity of the EDELWEISS-II dark matter search experiment together with short measurements at various other locations. Monitoring of the flux of thermal neutrons is accomplished using a mobile detection system with low background proportional counter filled with $^3$He. On average 75 neutrons per day are detected with a background level below 1 count per day (cpd). This provides a unique possibility of a day by day study of variations of the neutron field in a deep underground site. The measured average 4$pi$ neutron flux per cm$^{2}$ in the proximity of EDELWEISS-II is $Phi_{MB}=3.57pm0.05^{stat}pm0.27^{syst}times 10^{-6}$ neutrons/sec. We report the first experimental observation that the point-to-point thermal neutron flux at LSM varies by more than a factor two.