Using a sample of 43 bright (V<16.1, distance <13 kpc) RR Lyrae stars (RRLS) from the QUEST survey with spectroscopic radial velocities and metallicities, we find that several separate halo substructures contribute to the Virgo overdensity (VOD). While there is little evidence for halo substructure in the spatial distribution of these stars, their distribution in radial velocity reveals two moving groups. These results are reinforced when the sample is combined with a sample of blue horizontal branch stars that were identified in the SDSS, and the combined sample provides evidence for one additional moving group. These groups correspond to peaks in the radial velocity distribution of a sample of F type main-sequence stars that was recently observed in the same directon by SEGUE, although in one case the RRLS and F star groups may not lie at the same distance. One of the new substructures has a very narrow range in metallicity, which is more consistent with it being the debris from a destroyed globular cluster than from a dwarf galaxy. A small concentration of stars have radial velocities that are similar to the Virgo Stellar Stream (VSS) that was identified previously in a fainter sample of RRLS. Our results suggest that this feature extends to distances as short as ~12 kpc from its previous detection at ~19 kpc. None of the new groups and only one star in the sample have velocities that are consistent with membership in the leading tidal stream from the Sagittarius Dwarf Spheroidal Galaxy, which some authors have suggested is the origin of the VOD.