Spectroscopic observations of massive $z>7$ galaxies selected to have extremely large [OIII]+H$beta$ equivalent width (EW $sim1500$ r{A}) have recently revealed large Ly$alpha$ detection rates, in contrast to the weak emission seen in the general population. Why these systems are uniquely visible in Ly$alpha$ at redshifts where the IGM is likely significantly neutral is not clear. With the goal of better understanding these results, we have begun a campaign with MMT and Magellan to measure Ly$alpha$ in galaxies with similar [OIII]+H$beta$ EWs at $zsimeq2-3$. At these redshifts, the IGM is highly ionized, allowing us to clearly disentangle how the Ly$alpha$ properties depend on the [OIII]+H$beta$ EW. Here we present Ly$alpha$ EWs of $49$ galaxies at $z=2.2-3.7$ with intense [OIII]+H$beta$ line emission (EW $=300-3000$ r{A}). Our results demonstrate that strong Ly$alpha$ emission (EW $>20$ r{A}) becomes more common in galaxies with larger [OIII]+H$beta$ EW, reflecting a combination of increasingly efficient ionizing photon production and enhanced transmission of Ly$alpha$. Among the galaxies with the most extreme [OIII]+H$beta$ emission (EW $sim1500$ r{A}), we find that strong Ly$alpha$ emission is not ubiquitous, with only $50$ per cent of our population showing Ly$alpha$ EW $>20$ r{A}. Our data suggest that the range of Ly$alpha$ strengths is related to the observed ellipticity, with those systems that appear edge-on or elongated having weaker Ly$alpha$ emission. We use these results to interpret the anomalous Ly$alpha$ properties seen in $z>7$ galaxies with extreme [OIII]+H$beta$ emission and discuss implications for the escape of ionizing radiation from these extreme line emitting galaxies.