Several models of level densities exist and they often make simplified assumptions regarding the overall behavior of the total level densities (LD) and the intrinsic spin and parity distributions of the excited states. Normally, such LD models are constrained only by the measured $D_0$, i.e. the density of levels at the neutron separation energy of the compound nucleus (target plus neutron), and the sometimes subjective extrapolation of discrete levels. In this work we use microscopic Hartree-Fock-Bogoliubov (HFB) level densities, which intrinsically provide more realistic spin and parity distributions, and associate variations predicted by the HFB model with the observed double-differential cross sections at low outgoing neutron energy, region that is dominated by the LD input. With this approach we are able to perform fits of the LD based on actual experimental data, constraining the model and ensuring its consistency. This approach can be particularly useful in extrapolating the LD to nuclei for which high-excited discrete levels and/or values of $D_0$ are unknown. It also predicts inelastic gamma (n,n$^{prime}gamma$) cross sections that in some cases can differ significantly from more standard LD models such as Gilbert-Cameron.