Dusty galaxies at z ~ 2 span a wide range of relative brightness between rest-frame mid-infrared (8um) and ultraviolet wavelengths. We attempt to determine the physical mechanism responsible for this diversity. Dust-obscured galaxies (DOGs), which have rest-frame mid-IR to UV flux density ratios > 1000, might be abnormally bright in the mid-IR, perhaps due to prominent AGN and/or PAH emission, or abnormally faint in the UV. We use far-infrared data from the GOODS-Herschel survey to show that most DOGs with 10^12 L_Sun < L_IR < 10^13 L_Sun are not abnormally bright in the mid-IR when compared to other dusty galaxies with similar IR (8--1000um) luminosities. We observe a relation between the median IR to UV luminosity ratios and the median UV continuum power-law indices for these galaxies, and we find that only 24% have specific star formation rates which indicate the dominance of compact star-forming regions. This circumstantial evidence supports the idea that the UV- and IR-emitting regions in these galaxies are spatially coincident, which implies a connection between the abnormal UV faintness of DOGs and dust obscuration. We conclude that the range in rest-frame mid-IR to UV flux density ratios spanned by dusty galaxies at z ~ 2 is due to differing amounts of UV obscuration. Of galaxies with these IR luminosities, DOGs are the most obscured. We attribute differences in UV obscuration to either: 1) differences in the degree of alignment between the spatial distributions of dust and massive stars, or 2) differences in the total dust content.