Large clouds of cold atoms prepared in a magneto-optical trap are known to present spatiotemporal instabilities when the frequency of the trapping lasers is brought close to the atomic resonance. This system bears similarities with trapped plasmas where the role of the Coulomb interaction is played by the exchange of scattered photons, and astrophysical objects such as stars whose size is dependent on radiative forces. We present in this paper a study of the phase-space of such instabilities, and reveal different dynamical regimes. Three dimensional simulations of the highly nonlinear atomic dynamics permit a detailed analysis of the experimental observations.