Several experiments have been conducted in the YangYang Underground Laboratory in the Republic of Korea such as the search for dark matter and the search for neutrinoless double-beta decay, which require an extremely low background event rate due to the detector system and the environment. In underground experiments, neutrons have been identified as one of the background sources. The neutron flux in the experimental site needs to be determined to design a proper shielding system and for precise background estimation. We measured the neutron spectrum with a Bonner sphere spectrometer, with Helium-3 ($^{3}$He) proportional counters. The neutron flux at the underground laboratory was so low that the radioactive decays from the radioisotopes contained in the detector created a significant background interference to the neutron measurement. Using Monte Carlo simulations, the intrinsic $alpha$ background distribution due to the radioactive isotopes in the detector materials, was estimated. The neutron count rate of each Bonner sphere was measured from the pulse height spectrum of the $^{3}$He proportional counter, after subtracting the $alpha$ particle background. The neutron flux and the energy spectrum were determined using the unfolding technique. The total neutron flux measured was (4.46 $pm$ 0.66) $times$ $10^{-5}$ $rm{cm^{-2} s^{-1}}$, and the thermal and fast neutron flux (in the range 1 to 10 MeV) were (1.44 $pm$ 0.15) $times$ 10$^{-5}$ $rm{cm^{-2} s^{-1}}$ and (0.71 $pm$ 0.10) $times$ 10$^{-5}$ $rm{cm^{-2} s^{-1}}$, respectively.