Because all stars contribute to its gravitational potential, stellar clusters amplify perturbations collectively. In the limit of small fluctuations, this is described through linear response theory, via the so-called response matrix. While the evaluation of this matrix is somewhat straightforward for unstable modes (i.e. with a positive growth rate), it requires a careful analytic continuation for damped modes (i.e. with a negative growth rate). We present a generic method to perform such a calculation in spherically symmetric stellar clusters. When applied to an isotropic isochrone cluster, we recover the presence of a low-frequency weakly damped $ell = 1$ mode. We finally use a set of direct $N$-body simulations to test explicitly this prediction through the statistics of the correlated random walk undergone by a clusters density centre.