Temperature (5--250 K) and magnetic field (0--70 kOe) variations of the low-energy (1--10 meV) electrodynamics of spin excitations have been investigated for a complete set of light-polarization configurations for a ferroelectric magnet DyMnO$_3$ by using terahertz time-domain spectroscopy. We identify the pronounced absorption continuum (1--8 meV) with a peak feature around 2 meV, which is electric-dipole active only for the light $E$-vector along the a-axis. This absorption band grows in intensity with lowering temperature from the spin-collinear paraelectric phase above the ferroelectric transition, but is independent of the orientation of spiral spin plane ($bc$ or $ab$), as shown on the original $P_{rm s}$ (ferroelectric polarization) $parallel c$ phase as well as the magnetic field induced $P_{rm s}parallel a$ phase. The possible origin of this electric-dipole active band is argued in terms of the large fluctuations of spins and spin-current.