We explicitly construct every kinematically allowed three particle graviton-graviton-$P$ and photon-photon-$P$ S-matrix in every dimension and for every choice of the little group representation of the massive particle $P$. We also explicitly construct the spacetime Lagrangian that generates each of these couplings. In the case of gravitons we demonstrate that this Lagrangian always involves (derivatives of) two factors of the Riemann tensor, and so is always of fourth or higher order in derivatives. This result verifies one of the assumptions made in the recent preprint cite{Chowdhury:2019kaq} while attempting to establish the rigidity of the Einstein tree level S-matrix within the space of local classical theories coupled to a collection of particles of bounded spin.