The [OIII] 88 $mu$m fine structure emission line has been detected into the Epoch of Reionization (EoR) from star-forming galaxies at redshifts $6 < z lesssim 9$ with ALMA. These measurements provide valuable information regarding the properties of the interstellar medium (ISM) in the highest redshift galaxies discovered thus far. The [OIII] 88 $mu$m line observations leave, however, a degeneracy between the gas density and metallicity in these systems. Here we quantify the prospects for breaking this degeneracy using future ALMA observations of the [oiii] 52 $mu$m line. Among the current set of ten [OIII] 88 $mu$m emitters at $6 < z lesssim 9$, we forecast 52 $mu$m detections (at 6-$sigma$) in SXDF-NB1006-2, B14-6566, J0217-0208, and J1211-0118 within on-source observing times of 2-10 hours, provided their gas densities are larger than about $n_{mathrm{H}} gtrsim 10^2-10^3$ cm$^{-3}$. Other targets generally require much longer integration times for a 6-$sigma$ detection. Either successful detections of the 52 $mu$m line, or reliable upper limits, will lead to significantly tighter constraints on ISM parameters. The forecasted improvements are as large as $sim 3$ dex in gas density and $sim 1$ dex in metallicity for some regions of parameter space. We suggest SXDF-NB1006-2 as a promising first target for 52 $mu$m line measurements. We discuss how such measurements will help in understanding the mass metallicity relationship during the EoR.