Understanding and mitigating loss channels due to two-level systems (TLS) is one of the main cornerstones in the quest of realizing long photon lifetimes in superconducting quantum circuits. Typically, the TLS to which a circuit couples are modeled as a large bath without any coherence. Here we demonstrate that the coherence of TLS has to be considered to accurately describe the ring-down dynamics of a coaxial quarter-wave resonator with an internal quality factor of $0.5times10^9$ at the single-photon level. The transient analysis reveals effective non-Markovian dynamics of the combined TLS and cavity system, which we can accurately fit by introducing a comprehensive TLS model. The fit returns an average coherence time of around $T_2^*approx0.3,mathrm{mu s}$ for a total of $Napprox10^{9}$ TLS with power-law distributed coupling strengths. Despite the shortly coherent TLS excitations, we observe long-term effects on the cavity decay due to coherent elastic scattering between the resonator field and the TLS. Moreover, this model provides an accurate prediction of the internal quality factors temperature dependence.
Download