Primordial stars are likely to be very massive $geq30Msun$, form in isolation, and will likely leave black holes as remnants in the centers of their host dark matter halos in the mass range $10^{6}-10^{10}Ms$. Such early black holes, at redshifts z$gtsim10$, could be the seed black holes for the many supermassive black holes found in galaxies in the local universe. If they exist, their mergers with nearby supermassive black holes may be a prime signal for long wavelength gravitational wave detectors. We simulate formation of black holes in the center of high redshift dark matter halos and explore implications of initial natal kick velocities conjectured by some formation models. The central concentration of early black holes in present day galaxies is reduced if they are born even with moderate kicks of tens of km/s. The modest kicks allow the black holes to leave their parent halo, which consequently leads to dynamical friction being less effective on the lower mass black holes as compared to those still embedded in their parent halos. Therefore, merger rates may be reduced by more than an order of magnitude. Using analytical and illustrative cosmological N--body simulations we quantify the role of natal kicks of black holes formed from massive metal free stars on their merger rates with supermassive black holes in present day galaxies. Our results also apply to black holes ejected by the gravitational slingshot mechanism.