The effects of initially uniform magnetic fields on the formation and evolution of dense pillars and cometary globules at the boundaries of H II regions are investigated using 3D radiation-magnetohydrodynamics simulations. It is shown, in agreement with previous work, that a strong initial magnetic field is required to significantly alter the non-magnetised dynamics because the energy input from photoionisation is so large that it remains the dominant driver of the dynamics in most situations. Additionally it is found that for weak and medium field strengths an initially perpendicular field is swept into alignment with the pillar during its dynamical evolution, matching magnetic field observations of the `Pillars of Creation in M16 and also some cometary globules. A strong perpendicular magnetic field remains in its initial configuration and also confines the photoevaporation flow into a bar-shaped dense ionised ribbon which partially shields the ionisation front and would be readily observable in recombination lines. A simple analytic model is presented to explain the properties of this bright linear structure. These results show that magnetic field strengths in star-forming regions can in principle be significantly constrained by the morphology of structures which form at the borders of H II regions.