We take a closer look at the fundamental Casimir-Polder interaction between quantum particles and dispersive dielectric surfaces with surface polariton or plasmon resonances. Linear response theory shows that in the near field, van der Waals, regime the free energy shift of a particle contains a thermal component that depends exclusively on the population/excitation of the evanescent surface polariton/plasmon modes. Our work makes evident the link between particle surface interaction and near field thermal emission and demonstrates how this can be used to engineer Casimir-Polder forces. We also examine how the exotic effects of surface waves are washed out as the distance from the surface increases. In the case of molecules or excited state atoms, far field approximations result in a classical dipole-dipole interaction which depends on the surface reflectivity and the mean number of photons at the frequency of the atomic/molecular transition. Finally we present numerical results for the CP interaction between Cs atoms and various dielectric surfaces with a single polariton resonance and discuss the implications of temperature and retardation effects for specific spectroscopic experiments.