The microscopic description of neutron scattering by $^{16}$O below 30 MeV is carried out by means of the continuum particle-vibration coupling (cPVC) method with the Skyrme nucleon-nucleon ($NN$) effective interaction. In the cPVC method, a proper boundary condition on a nucleon in continuum states is imposed, which enables one to evaluate the transition matrix in a straightforward manner. Experimental data of the total and total-elastic cross sections are reproduced quite well by the cPVC method. An important feature of the result is the fragmentation of the single-particle resonance into many peaks as well as the shift of its centroid energy. Thus, some part of the fine structure of the experimental cross sections at lower energies is well described by the cPVC framework. The cPVC method based on a real $NN$ effective interaction is found to successfully explain about 85% of the reaction cross section, through explicit channel-coupling effects.