Galaxies represent one of the preferred candidate sources to drive the reionization of the universe. Even as gains are made in mapping the galaxy UV luminosity density to z>6, significant uncertainties remain regarding the conversion to the implied ionizing emissivity. The relevant unknowns are the Lyman-continuum (LyC) photon production efficiency xi_{ion} and the escape fraction f_{esc}. As we show here, the first of these unknowns is directly measureable in z=4-5 galaxies, based on the impact the Halpha line has on the observed IRAC fluxes. By computing a LyC photon production rate from the implied Halpha luminosities for a broad selection of z=4-5 galaxies and comparing this against the dust-corrected UV-continuum luminosities, we provide the first-ever direct estimates of the LyC photon production efficiency xi_{ion} for the z>~4 galaxy population. We find log_{10} xi_{ion}/[Hz/ergs] to have a mean value of 25.27_{-0.03}^{+0.03} and 25.34_{-0.02}^{+0.02} for sub-L* z=4-5 galaxies adopting Calzetti and SMC dust laws, respectively. Reassuringly, both values are consistent with standardly assumed xi_{ion}s in reionization models, with a slight preference for higher xi_{ion}s (by ~0.1 dex) adopting the SMC dust law. A modest ~0.03-dex increase in these estimates would result if the escape fraction for ionizing photons is non-zero and galaxies dominate the ionizing emissivity at z~4.4. High values of xi_{ion} (~25.5-25.8 dex) are derived for the bluest galaxies (beta<-2.3) in our samples, independent of dust law and consistent with results for a z=7.045 galaxy. Such elevated values of xi_{ion} would have important consequences, indicating that f_{esc} cannot be in excess of 13% unless the galaxy UV luminosity function does not extend down to -13 mag or the clumping factor is greater than 3. A low escape fraction would fit well with the low rate of LyC leakage observed at z~3.