We show that Out(G) is residually finite if G is a one-ended group that is hyperbolic relative to virtually polycyclic subgroups. More generally, if G is one-ended and hyperbolic relative to proper residually finite subgroups, the group of outer automorphisms preserving the peripheral structure is residually finite. We also show that Out(G) is virtually p-residually finite for every prime p if G is one-ended and toral relatively hyperbolic, or infinitely-ended and virtually p-residually finite.