In this Letter I use the variation of the spread in rotation measure (RM) with Galactic latitude to separate the Galactic from the extragalactic contributions to RM. This is possible since the latter does not depend on Galactic latitude. As input data I use RMs from the catalogue by Taylor, Stil, and Sunstrum, supplemented with published values for the spread in RM (`sigmaRM) in specific regions on the sky. I test 4 models of the free electron column density (which I will abbreviate to `DMinf) of the Milky Way, and the best model builds up DMinf on a characteristic scale of a few kpc from the Sun. sigmaRM correlates well with DMinf. The measured sigmaRM can be modelled as a Galactic contribution, consisting of a term sigmaRM,MW that is amplified at smaller Galactic latitudes as 1/sin|b|, in a similar way to DMinf, and an extragalactic contribution, sigmaRM,EG, that is independent of latitude. This model is sensitive to the relative magnitudes of sigmaRM,MW and sigmaRM,EG, and the best fit is produced by sigmaRM,MW approx. 8 rad/m^2 and sigmaRM,EG approx. 6 rad/m^2. The 4 published values for sigmaRM as a function of latitude suggest an even larger sigmaRM,MW contribution and a smaller sigmaRM,EG. This result from the NVSS RMs and published sigmaRM shows that the Galactic contribution dominates structure in RM on scales between about 1degr -- 10degr on the sky. I work out which factors contribute to the variation of sigmaRM with Galactic latitude, and show that the sigmaRM,EG I derived is an upper limit. Furthermore, to explain the modelled sigmaRM,MW requires that structure in <B||> has a 1-sigma spread <~ 0.4 microG.