The antiferromagnetic Ruddlesden-Popper ruthenate Ca$_3$Ru$_2$O$_7$ is a model polar metal, combining inversion symmetry breaking with metallic conductivity; however, its low temperature ($T < 48$ K) crystal structure and Fermi surface topology remain ambiguous despite numerous measurements and theoretical studies. Here we perform both first principles calculations with static correlations and angle resolved photoelectron spectroscopy experiments to construct a complete model of Ca$_3$Ru$_2$O$_7$, reconciling inconsistencies among interpretations of electrical transport, thermopower measurements, and momentum- and energy-resolved band dispersions. The solution relies on treating the interplay among Coulomb repulsion, magnetic ordering, spin-orbit interactions, and the RuO$_6$ octahedral degrees-of-freedom on equal footing. For temperatures $30<T < 48$ K, we propose weak electron-electron interactions produce a symmetry-preserving metal-semimetal transition with Weyl nodes in proximity to the Fermi level, whereas a new orthorhombic $Pn2_1a$ structure emerges for $T<30$ K, exhibiting charge and spin density waves from enhanced Coulombic interactions.