Phase transitions have recently been formulated in the time domain of quantum many-body systems, a phenomenon dubbed dynamical quantum phase transitions (DQPTs), whose phenomenology is often divided in two types. One refers to distinct phases according to long-time averaged order parameters, while the other is focused on the non-analytical behavior emerging in the rate function of the Loschmidt echo. Here we show that such DQPTs can be found in systems with few degrees of freedom, i.e. they can take place without resorting to the traditional thermodynamic limit. We illustrate this by showing the existence of the two types of DQPTs in a quantum Rabi model -- a system involving a spin-$frac{1}{2}$ and a bosonic mode. The dynamical criticality appears in the limit of an infinitely large ratio of the spin frequency with respect to the bosonic one. We determine its dynamical phase diagram and study the long-time averaged order parameters, whose semiclassical approximation yields a jump at the transition point. We find the critical times at which the rate function becomes non-analytical, showing its associated critical exponent as well as the corrections introduced by a finite frequency ratio. Our results open the door for the study of DQPTs without the need to scale up the number of components, thus allowing for their investigation in well controllable systems.