A luminous X-ray source is associated with a cluster (MGG-11) of young stars ~200pc from the center of the starburst galaxy M82. The properties of the X-ray source are best explained by a black hole with a mass of at least 350Msun, which is intermediate between stellar-mass and supermassive black holes. A nearby but somewhat more massive star cluster (MGG-9) shows no evidence of such an intermediate mass black hole, raising the issue of just what physical characteristics of the clusters can account for this difference. Here we report numerical simulations of the evolution and the motions of stars within the clusters, where stars are allowed to mergers with each other. We find that for MGG-11 dynamical friction leads to the massive stars sinking rapidly to the center of the cluster to participate in a runaway collision, thereby producing a star of 800-3000Msun, which ultimately collapses to an black hole of intermediate mass. No such runaway occurs in the cluster MGG-9 because the larger cluster radius leads to a mass-segregation timescale a factor of five longer than for MGG-11.