The fraction of ionizing photons (fesc) that escape from z>6 galaxies is an important parameter when assessing the role of these objects in the reionization of the Universe, but the opacity of the intergalactic medium precludes a direct measurement of fesc for individual galaxies at these epochs. We argue, that since fesc regulates the impact of nebular emission on the spectra of galaxies, it should nonetheless be possible to indirectly probe fesc well into the reionization epoch. As a first step, we demonstrate that by combining measurements of the rest-frame UV slope beta with the equivalent width of the Hb emission line, galaxies with very high Lyman continuum escape fractions (fesc>0.5) should be identifiable up to z~9 through spectroscopy with the upcoming James Webb Space Telescope (JWST). By targeting strongly lensed galaxies behind low-redshift galaxy clusters, JWST spectra of sufficiently good quality can be obtained for M(1500)<-16.0 galaxies at z~7 and for M(1500)<-17.5 galaxies at z~9. Dust-obscured star formation may complicate the analysis, but supporting observations with ALMA or the planned SPICA mission may provide useful constraints on the dust properties of these galaxies.