We present high-resolution (0.3) ALMA 870um imaging of 52 sub-millimeter galaxies (SMGs) in the Ultra Deep Survey (UDS) field and investigate the size and morphology of the sub-millimeter (sub-mm) emission on 2-10kpc scales. We derive a median intrinsic angular size of FWHM=0.30$pm$0.04 for the 23 SMGs in the sample detected at a signal-to-noise ratio (SNR) >10. Using the photometric redshifts of the SMGs we show that this corresponds to a median physical half-light diameter of 2.4$pm$0.2kpc. A stacking analysis of the SMGs detected at an SNR <10 shows they have sizes consistent with the 870um-bright SMGs in the sample. We compare our results to the sizes of SMGs derived from other multi-wavelength studies, and show that the rest-frame ~250um sizes of SMGs are consistent with studies of resolved 12CO (J=3-2 to 7-6) emission lines, but that sizes derived from 1.4GHz imaging appear to be approximately two times larger on average, which we attribute to cosmic ray diffusion. The rest-frame optical sizes of SMGs are around four times larger than the sub-millimeter sizes, indicating that the star formation in these galaxies is compact relative to the pre-existing stellar distribution. The size of the starburst region in SMGs is consistent with the majority of the star formation occurring in a central region, a few kpc in extent, with a median star formation rate surface density of 90$pm$30Msol/yr/kpc$^2$, which may suggest that we are witnessing an intense period of bulge growth in these galaxies.