We develop an elasto-plastic description for the transient dynamics prior to steady flow of athermally yielding materials. Our mean-field model not only reproduces the experimentally observed non-linear time dependence of the shear-rate response to an external shear-stress, but also allows for the determination of the different physical processes involved in the onset of the re-acceleration phase after the initial critical slowing down and a distinct well defined fluidization phase. The evidenced power-law dependence of the fluidization time on the distance of the applied to an age dependent static yield stress is not universal but strongly dependent on initial conditions.