We investigate the connection between globular clusters and ultra-compact dwarf galaxies (UCDs) by examining the properties of several compact objects associated with M87, all of which were previously classified as globular clusters. Combining imaging from the Hubble Space Telescope with ground-based Keck spectroscopy, we find two objects to have half-light radii, velocity dispersions and mass-to-light ratios that are consistent with the predictions of population synthesis models for old, metal-rich, luminous globular clusters. Three other objects are much larger, with half-light radii of approximately 20pc, and have V-band mass-to-light ratios in the range 6-9. These objects, which we consider to be UCDs, resemble the nuclei of nucleated dwarf elliptical galaxies in Virgo, having similar mass-to-light ratios, luminosities and colors. These UCDs are found to obey the extrapolated scaling relations of galaxies more closely than those of Galactic globular clusters. There appears to be a transition between the two types of stellar systems at a mass of about two million solar masses. If the UCDs are gravitationally bound, then we suggest that the presence of dark matter is the fundamental property distinguishing globular clusters from UCDs. More than half of the UCD candidates uncovered in the ACS Virgo Cluster Survey are associated with a single galaxy -- M87 -- which suggests that proximity to the Virgo center may be of critical importance for the formation of these objects. These results show that distinguishing bonafide UCDs from bright globular clusters requires a careful analysis of their detailed structural and dynamical properties, particularly their mass-to-light ratios. (ABRIDGED)