This is the fourth in a series of papers studying the astrophysics and cosmology of massive, dynamically relaxed galaxy clusters. Here, we use measurements of weak gravitational lensing from the Weighing the Giants project to calibrate Chandra X-ray measurements of total mass that rely on the assumption of hydrostatic equilibrium. This comparison of X-ray and lensing masses provides a measurement of the combined bias of X-ray hydrostatic masses due to both astrophysical and instrumental sources. Assuming a fixed cosmology, and within a characteristic radius (r_2500) determined from the X-ray data, we measure a lensing to X-ray mass ratio of 0.96 +/- 9% (stat) +/- 9% (sys). We find no significant trends of this ratio with mass, redshift or the morphological indicators used to select the sample. In accordance with predictions from hydro simulations for the most massive, relaxed clusters, our results disfavor strong, tens-of-percent departures from hydrostatic equilibrium at these radii. In addition, we find a mean concentration of the sample measured from lensing data of c_200 = $3.0_{-1.8}^{+4.4}$. Anticipated short-term improvements in lensing systematics, and a modest expansion of the relaxed lensing sample, can easily increase the measurement precision by 30--50%, leading to similar improvements in cosmological constraints that employ X-ray hydrostatic mass estimates, such as on Omega_m from the cluster gas mass fraction.