We reveal an intriguing manifestation of topology, which appears in the depletion rate of topological states of matter in response to an external drive. This phenomenon is presented by analyzing the response of a generic 2D Chern insulator subjected to a circular time-periodic perturbation: due to the systems chiral nature, the depletion rate is shown to depend on the orientation of the circular shake. Most importantly, taking the difference between the rates obtained from two opposite orientations of the drive, and integrating over a proper drive-frequency range, provides a direct measure of the topological Chern number of the populated band ($ u$): this differential integrated rate is directly related to the strength of the driving field through the quantized coefficient $eta_0!=! u /hbar^2$. Contrary to the integer quantum Hall effect, this quantized response is found to be non-linear with respect to the strength of the driving field and it explicitly involves inter-band transitions. We investigate the possibility of probing this phenomenon in ultracold gases and highlight the crucial role played by edge states in this effect. We extend our results to 3D lattices, establishing a link between depletion rates and the non-linear photogalvanic effect predicted for Weyl semimetals. The quantized circular dichroism revealed in this work designates depletion-rate measurements as a universal probe for topological order in quantum matter.