Giant lobes of plasma extend 55 degrees above and below the Galactic Center, glowing in emission from gamma rays (the Fermi Bubbles) to microwaves (the WMAP haze) and polarized radio waves. We use ultraviolet absorption-line spectra from the Hubble Space Telescope to constrain the velocity of the outflowing gas within these regions, targeting the quasar PDS 456 (Galactic coordinates l,b=10.4, +11.2 degrees). This sightline passes through a clear biconical structure seen in hard X-ray and gamma-ray emission near the base of the northern Fermi Bubble. We report two high-velocity metal absorption components, at v_LSR=-235 and +250 km/s, which cannot be explained by co-rotating gas in the Galactic disk or halo. Their velocities are suggestive of an origin on the front and back side of an expanding biconical outflow emanating from the Galactic Center. We develop simple kinematic biconical outflow models that can explain these observed profiles with an outflow velocity of ~900 km/s and a full opening angle of ~110 degrees (matching the X-ray bicone). This indicates Galactic Center activity over the last ~2.5-4.0 Myr, in line with age estimates of the Fermi Bubbles. The observations illustrate the use of UV absorption-line spectroscopy to probe the properties of swept-up gas venting into the Fermi Bubbles.