The fundamental, or first, band gap is of unmatched importance in the study of photonic crystals. Here, we address precisely where this gap can be opened in the band structure of three-dimensional photonic crystals. Although strongly constrained by symmetry, this problem cannot be addressed directly with conventional band-symmetry analysis due to the existence of a photonic polarization vortex at zero frequency. We develop an approach for overcoming the associated symmetry singularity by incorporating fictitious, auxiliary longitudinal modes. Our strategy also enables us to extend recent developments in symmetry-based topological analysis to the fundamental gap of three-dimensional photonic crystals. Exploiting this, we systematically study the topology of the minimal fundamental gaps. This reveals the existence of topological gap-obstructions that push the fundamental gap higher than what a conventional analysis would suggest. Our work demonstrates that topology can play a crucial role in the opening of the fundamental photonic gap and informs future theoretical and experimental searches for conventional and topological band gaps in three-dimensional photonic crystals.