The long-ranged nature of the Coulomb potential requires a proper accounting for the influence of even distant electrostatic boundaries in the determination of the solvation free energy of a charged solute. We introduce an exact rewriting of the free energy change upon charging a solute that explicitly isolates the contribution from these boundaries and quantifies the impact of the different boundaries on the free energy. We demonstrate the importance and advantages of appropriately referencing the electrostatic potential to that of the vacuum through the study of several simple model charge distributions, for which we can isolate an analytic contribution from the boundaries that can be readily evaluated in computer simulations of molecular systems. Finally, we highlight that the constant potential of the bulk dielectric phase - the Bethe potential - cannot contribute to the solvation thermodynamics of a single charged solute when the charge distributions of the solvent and solute do not overlap in relevant configurations. But when the charge distribution of a single solute can overlap with the intramolecular charge distribution of solvent molecules, as is the case in electron holography, for example, the Bethe potential is needed when comparing to experiment. Our work may also provide insight into the validity of extra thermodynamic assumptions traditionally made during the experimental determination of single ion solvation free energies.