Motivated by recent experiments demonstrating intricate quantum Hall physics on the surface of elemental bismuth, we consider proximity coupling an $s$-wave superconductor to a two-dimensional electron gas with strong Rashba spin-orbit interactions in the presence of a strong perpendicular magnetic field. We focus on the high-field limit so that the superconductivity can be treated as a perturbation to the low-lying Landau levels. In the clean case, wherein the superconducting order parameter takes the form of an Abrikosov vortex lattice, we show that a lattice of hybridized Majorana modes emerges near the plateau transition of the lowest Landau level. However, unless magnetic-symmetry-violating perturbations are present, the system always has an even number of chiral Majorana edge modes and thus is strictly speaking Abelian in nature, in agreement with previous work on related setups. Interestingly, however, a weak topological superconducting phase can very naturally be stabilized near the plateau transition for the square vortex lattice. The relevance of our findings to potential near-term experiments on proximitized materials such as bismuth will be discussed.