We consider a classical fermion and a classical scalar, propagating on two different kinds of 4-dimensional diffeomorphism breaking gravity backgrounds, and we derive the one-loop effective dispersion relation for matter, after integrating out gravitons. One gravity model involves quadratic divergences at one-loop, as in Einstein gravity, and the other model is the $z=3$ non-projectable Horava-Lifshitz gravity, which involves logarithmic divergences only. Although these two models behave differently in the UV, the IR phenomenology for matter fields is comparable: {it(i)} for generic values for the parameters, both models identify $10^{10}$ GeV as the typical characteristic scale above which they are not consistent with current upper bounds on Lorentz symmetry violation; {it(ii)} on the other hand there is always, for both models, a fine-tuning of parameters which allows the cancellation of the indicator for Lorentz symmetry violation.