The Coulomb phase, with its dipolar correlations and pinch-point-scattering patterns, is central to discussions of geometrically frustrated systems, from water ice to binary and mixed-valence alloys, as well as numerous examples of frustrated magnets. The emergent Coulomb phase of lattice-based systems has been associated with divergence-free fields and the absence of long-range order. Here, we go beyond this paradigm, demonstrating that a Coulomb phase can emerge naturally as a persistent fluctuating background in an otherwise ordered system. To explain this behavior, we introduce the concept of the fragmentation of the field of magnetic moments into two parts, one giving rise to a magnetic monopole crystal, the other a magnetic fluid with all the characteristics of an emergent Coulomb phase. Our theory is backed up by numerical simulations, and we discuss its importance with regard to the interpretation of a number of experimental results.