We describe a moderate-resolution FUSE mini-survey of H2 in the Milky Way and Magellanic Clouds, using four hot stars and four AGN as background sources. FUSE spectra of nearly every stellar and extragalactic source exhibit numerous absorption lines from the H2 Lyman and Werner bands between 912 and 1120 A. One extragalactic sightline, PKS 2155-304, with low N(HI) shows no detectable H2 and could be the Lockman Hole of molecular gas, of importance for QSO absorption-line studies. We measure H2 column densities in low rotational states (J = 0 and 1) to derive rotational and/or kinetic temperatures of diffuse interstellar gas. The higher-J abundances can constrain models of the UV radiation fields and gas densities. In three optically thick clouds toward extragalactic sources, we find n(H) ~ 30-50 cm(-3) and cloud thicknesses of 2-3 pc. The rotational temperatures for H2 at high Galactic latitude, <T_01> = 107 +/- 17 K (seven sightlines) and 120 +/- 13 K (three optically thick clouds), are higher than those in the Copernicus sample composed primarily of targets in the disk. We find no evidence for great differences in the abundance or state of excitation of H2 between sight lines in the Galaxy and those in the SMC and LMC. In the future, we will probe the distribution and physical parameters of diffuse molecular gas in the disk and halo and in the lower-metallicity environs of the LMC and SMC.