Galaxy clusters are expected to form hierarchically in a LCDM universe, growing primarily through mergers with lower mass clusters and the continual accretion of group-mass halos. Galaxy clusters assemble late, doubling their masses since z~0.5, and so the outer regions of clusters should be replete with infalling group-mass systems. We present an XMM-Newton survey to search for X-ray groups in the infall regions of 23 massive galaxy clusters at z~0.2, identifying 39 X-ray groups that have been spectroscopically confirmed to lie at the cluster redshift. These groups have mass estimates in the range 2x10^13-7x10^14Msun, and group-to-cluster mass ratios as low as 0.02. The comoving number density of X-ray groups in the infall regions is ~25x higher than that seen for isolated X-ray groups from the XXL survey. The average mass per cluster contained within these X-ray groups is 2.2x10^14Msun, or 19% of the mass within the primary cluster itself. We estimate that ~10^15Msun clusters increase their masses by 16% between z=0.223 and the present day due to the accretion of groups with M200>10^13.2Msun. This represents about half of the expected mass growth rate of clusters at these late epochs. The other half is likely to come from smooth accretion of matter not bound in halos. The mass function of the infalling X-ray groups appears significantly top-heavy with respect to that of field X-ray systems, consistent with expectations from numerical simulations, and the basic consequences of collapsed massive dark matter halos being biased tracers of the underlying large-scale density distribution.