We study the effect of an ancillary system on the quantum speed limit time in different non-Markovian environments. Through employing an ancillary system coupled with the quantum system of interest via hopping interaction and investigating the cases that both the quantum system and ancillary system interact with their independent/common environment, and the case that only the system of interest interacts with the environment, we find that the quantum speed limit time will become shorter with enhancing the interaction between the system and environment and show periodic oscillation phenomena along with the hopping interaction between the quantum system and ancillary system increasing. The results indicate that the hopping interaction with the ancillary system and the structure of environment determine the degree of which the evolution of the quantum system can be accelerated.