The understanding of the shrinkage dynamics of plasmonic bubbles formed around metallic nanoparticles immersed in liquid and irradiated by a resonant light source is crucial for the usage of these bubbles in numerous applications. In this paper we experimentally show and theoretically explain that a plasmonic bubble during its shrinkage undergoes two different phases: first, a rapid partial bubble shrinkage governed by vapor condensation and, second, a slow diffusion-controlled bubble dissolution. The history of the bubble formation plays an important role in the shrinkage dynamics during the first phase, as it determines the gas-vapor ratio in the bubble composition. Higher laser powers lead to more vaporous bubbles, while longer pulses and higher dissolved air concentrations lead to more gaseous bubbles. The dynamics of the second phase barely depends on the history of bubble formation, i.e. laser power and pulse duration, but strongly on the dissolved air concentration, which defines the concentration gradient at the bubble interface. Finally, for the bubble dissolution in the second phase, with decreasing dissolved air concentration, we observe a gradual transition from a $R(t) propto (t_0 - t) ^{1/3}$ scaling law to a $R(t) propto (t_0 - t) ^{1/2}$ scaling law, where $t_0$ is the lifetime of the bubble and theoretically explain this transition.