Tidal effects have an important impact on the late inspiral of compact binary systems containing neutron stars. Most current models of tidal deformations of neutron stars assume that the tidal bulge is directly related to the tidal field generated by the companion, with a constant response coefficient. However, if the orbital motion approaches a resonance with one of the internal modes of the neutron star, this adiabatic description of tidal effects starts to break down, and the tides become dynamical. In this paper, we consider dynamical tides in general relativity due to the quadrupolar fundamental oscillation mode of a neutron star. We devise a description of the effects of the neutron stars finite size on the orbital dynamics based on an effective point-particle action augmented by dynamical quadrupolar degrees of freedom. We analyze the post-Newtonian and test-particle approximations of this model and incorporate the results into an effective-one-body Hamiltonian. This enables us to extend the description of dynamical tides over the entire inspiral. We demonstrate that dynamical tides give a significant enhancement of matter effects compared to adiabatic tides, at least for neutron stars with large radii and for low mass-ratio systems, and should therefore be included in accurate models for gravitational-wave data analysis.