We use Hubble Space Telescope imaging to measure the absolute proper motion of the hypervelocity star (HVS) HE 0437-5439, a short-lived B star located in the direction of the Large Magellanic Cloud (LMC). We observe (mu_alpha, mu_delta)=(+0.53+-0.25(stat)+-0.33(sys), +0.09+-0.21(stat)+-0.48(sys)) mas/yr. The velocity vector points directly away from the center of the Milky Way; an origin from the center of the LMC is ruled out at the 3-sigma level. The flight time of the HVS from the Milky Way exceeds its main-sequence lifetime, thus its stellar nature requires it to be a blue straggler. The large space velocity rules out a Galactic-disk ejection. Combining the HVSs observed trajectory, stellar nature, and required initial velocity, we conclude that HE 0437-5439 was most likely a compact binary ejected by the Milky Ways central black hole.