We apply the Picard and Magnus expansions to both the semiclassical and the quantum Rabi model, with a switchable matter-field coupling. The case of the quantum Rabi model ia a paradigmatic example of finite-time quantum electrodynamics (QED), and in this case we build an intuitive diagrammatic representation of the Picard series. In particular, we show that regular oscillations in the mean number of photons, ascribed to the dynamical Casimir effect (DCE) for the the generation of photons and to the anti-DCE for their destruction, take place at twice the resonator frequency $omega$. Such oscillations, which are a clear dynamical smoking gun of the ultrastrong coupling regime, can be predicted by first-order Picard expansion. We also show that the Magnus expansion can be used, through concatenation, as an efficient numerical integrator for both the semiclassical and the quantum Rabi model. In the first case, we find distinctive features in the Fourier spectrum of motion, with a single peak at the Rabi frequency $Omega$ and doublets at frequencies $2nomegapmOmega$, with $n$ positive integer. We explain these doublets, which are a feature beyond the rotating wave approximation (RWA), on the basis of the Picard series.