We study theoretically the deterministic dynamics of single-domain ferromagnetic nanoparticles in dilute ferrofluids, which is induced by a time-varying gradient magnetic field. Using the force and torque balance equations, we derive a set of the first-order differential equations describing the translational and rotational motions of such particles characterized by small Reynolds numbers. Since the gradient magnetic field generates both the translations and rotations of particles, these motions are coupled. Based on the derived set of equations, we demonstrate this fact explicitly by expressing the particle position through the particle orientation angle, and vice versa. The obtained expressions are used to show that the solution of the basic set of equations is periodic in time and to determine the intervals, where the particle coordinate and orientation angle oscillate. In addition, this set of equations is solved approximately for the case of small characteristic frequency of the particle oscillations. With this condition, we find that all particles perform small translational oscillations near their initial positions. In contrast, the orientation angle oscillates near the initial angle only if particles are located in the vicinity of zero point of the gradient magnetic field. The possible applications of the obtained results in biomedicine and separation processes are also discussed.