We benchmark the decoherence of superconducting qubits to examine the temporal stability of energy-relaxation and dephasing. By collecting statistics during measurements spanning multiple days, we find the mean parameters $overline{T_{1}}$ = 49 $mu$s and $overline{T_{2}^{*}}$ = 95 $mu$s, however, both of these quantities fluctuate explaining the need for frequent re-calibration in qubit setups. Our main finding is that fluctuations in qubit relaxation are local to the qubit and are caused by instabilities of near-resonant two-level-systems (TLS). Through statistical analysis, we determine switching rates of these TLS and observe the coherent coupling between an individual TLS and a transmon qubit. Finally, we find evidence that the qubits frequency stability is limited by capacitance noise. Importantly, this produces a 0.8 ms limit on the pure dephasing which we also observe. Collectively, these findings raise the need for performing qubit metrology to examine the reproducibility of qubit parameters, where these fluctuations could affect qubit gate fidelity.