We study the renormalization of a non-magnetic impuritys scattering potential due to the presence of a massless collective spin mode at a ferromagnetic quantum critical point. To this end, we compute the lowest order vertex corrections in two- and three-dimensional systems, for arbitrary scattering angle and frequency of the scattered fermions, as well as band curvature. We show that only for backward scattering in D=2 does the lowest order vertex correction diverge logarithmically in the zero frequency limit. In all other cases, the vertex corrections approach a finite (albeit possibly large) value in the zero frequency limit. We demonstrate that vertex corrections are strongly suppressed with increasing curvature of the fermionic bands. Moreover, we show how the frequency dependence of vertex corrections varies with the scattering angle. We also discuss the form of higher order ladder vertex corrections and show that they can be classified according to the zero-frequency limit of the lowest order vertex correction. We show that even in those cases where the latter is finite, summing up an infinite series of ladder vertex diagrams can lead to a strong enhancement (or divergence) of the impuritys scattering potential. Finally, we suggest that the combined frequency and angular dependence of vertex corrections might be experimentally observable via a combination of frequency dependent and local measurements, such as scanning tunneling spectroscopy on ordered impurity structures, or measurements of the frequency dependent optical conductivity.