We explore the electromagnetic contribution to the charge symmetry breaking in the octet baryon masses using a subtracted dispersion relation based on the Cottingham formula. For the proton-neutron mass splitting we report a minor revision of the recent analysis of Walker-Loud, Carlson and Miller. For the electromagnetic structure of the hyperons we constrain our analysis, where possible, by a combination of lattice QCD and SU(3) symmetry breaking estimates. The results for the baryon mass splittings are found to be compatible with recent lattice QCD+QED determinations. The uncertainties in the dispersive analysis are dominated by the lack of knowledge of the hyperon inelastic structure.