Recent work suggests that the first generation of stars, the so-called Population III (Pop III), could have formed primarily in binaries or as members of small multiple systems. Here we investigate the impact of X-ray feedback from High-Mass X-ray Binaries (HMXBs) left behind in stellar binary systems after the primary forms a black hole (BH), accreting gas at a high rate from the companion, a process that is thought to be favored at the low metallicities characteristic of high-redshift gas. Thanks to their large mean free path, X-rays are capable of preionizing and preheating the gas in the intergalactic medium (IGM) and in haloes long before the reionization of the Universe is complete, and thus could have strongly affected the formation of subsequent generations of stars as well as reionization. We have carried out zoomed hydrodynamical cosmological simulations of minihaloes, accounting for the formation of Pop III stars and their collapse into BHs and HMXBs, and the associated radiation-hydrodynamic feedback from UV and X-ray photons. We find no strong net feedback from HMXBs on the simulated star formation history. On the other hand, the preheating of the IGM by HMXBs leads to a strong suppression of small-scale structures and significantly lowers the recombination rate in the IGM, thus yielding a net positive feedback on reionization. We further show that X-ray feedback from HMXBs can augment the ionizing feedback from the Pop III progenitor stars to suppress gas accretion onto the first BHs, limiting their growth into supermassive BHs. Finally, we show that X-ray ionization by HMXBs leaves distinct signatures in the properties of the high-redshift hydrogen that may be probed in upcoming observations of the redshifted 21cm spin-flip line.