The nature of the ultra-luminous X-ray sources (ULXs) in the nearby galaxies is a matter of debates. One of the popular hypothesis associates them with accretion at a sub-Eddington rate on to intermediate mass black holes. Another possibility is a st
ellar-mass black hole in a high-mass X-ray binary accreting at super-Eddington rates. In this paper we find a highly significant association between brightest X-ray sources in the Antennae galaxies and stellar clusters. On the other hand, we show that most of the X-ray sources are located outside of these clusters. We study clusters associated with the ULXs using the ESO Very Large Telescope spectra and the Hubble Space Telescope data together with the theoretical evolutionary tracks and determine their ages to be below 6 Myr. This implies that the ULX progenitor masses certainly exceed 30 and for some objects are closer to 100 solar masses. We also estimate the ages of clusters situated close to the less luminous X-ray sources (with luminosity in the range 3x10^38 < L_X < 10^39 erg/s) and find that most of them are younger than 10 Myr, because they are surrounded by strong H$alpha$ emission. These findings are consistent with the idea that majority of ULXs are massive X-ray binaries that have been ejected in the process of formation of stellar clusters by a few-body encounters and at the same time rules out the proposal that most of the ULXs are intermediate mass black holes.
