We report on the low frequency (LF) noise measurements in magnetic tunnel junctions (MTJs) below 4 K and at low bias, where the transport is strongly affected by scattering with magnons emitted by hot tunnelling electrons, as thermal activation of magnons from the environment is suppressed. For both CoFeB/MgO/CoFeB and CoFeB/AlO$_{x}$/CoFeB MTJs, enhanced LF noise is observed at bias voltage around magnon emission energy, forming a peak in the bias dependence of noise power spectra density, independent of magnetic configurations. The noise peak is much higher and broader for unannealed AlO$_{x}$-based MTJ, and besides Lorentzian shape noise spectra in the frequency domain, random telegraph noise (RTN) is visible in the time traces. During repeated measurements the noise peak reduces and the RTN becomes difficult to resolve, suggesting defects being annealed. The Lorentzian shape noise spectra can be fitted with bias-dependent activation of RTN, with the attempt frequency in the MHz range, consistent with magnon dynamics. These findings suggest magnon-assisted activation of defects as the origin of the enhanced LF noise.