We present Suzaku X-ray observations along two edge regions of the Fermi Bubbles, with eight ~20 ksec pointings across the northern part of the North Polar Spur (NPS) surrounding the north bubble and six across the southernmost edge of the south bubble. After removing compact X-ray features, diffuse X-ray emission is clearly detected and is well reproduced by a three-component spectral model consisting of unabsorbed thermal emission (temperature kT ~0.1 keV from the Local Bubble (LB), absorbed kT ~0.3 keV thermal emission related to the NPS and/or Galactic Halo (GH), and a power-law component at a level consistent with the cosmic X-ray background. The emission measure (EM) of the 0.3 keV plasma decreases by ~50% toward the inner regions of the north-east bubble, with no accompanying temperature change. However, such a jump in the EM is not clearly seen in the south bubble data. While it is unclear if the NPS originates from a nearby supernova remnant or is related to previous activity within/around the Galactic Center, our Suzaku observations provide evidence suggestive of the latter scenario. In the latter framework, the presence of a large amount of neutral matter absorbing the X-ray emission as well as the existence of the kT ~ 0.3 keV gas can be naturally interpreted as a weak shock driven by the bubbles expansion in the surrounding medium, with velocity v_exp ~300 km/s (corresponding to shock Mach number M ~1.5), compressing the GH gas to form the NPS feature. We also derived an upper limit for any non-thermal X-ray emission component associated with the bubbles and demonstrate, that in agreement with the findings above, the non-thermal pressure and energy estimated from a one-zone leptonic model of its broad-band spectrum, are in rough equilibrium with that of the surrounding thermal plasma.