We estimate the galaxy density along lines of sight to hard extragalactic gamma-ray sources by correlating source positions on the sky with a void catalog based on the Sloan Digital Sky Survey (SDSS). Extragalactic gamma-ray sources that are detected at very high energy (VHE; E>100 GeV) or have been highlighted as VHE-emitting candidates in the Fermi Large Area Telescope hard source catalog (together referred to as VHE-like sources) are distributed along underdense lines of sight at the 2.4 sigma level. There is also a less suggestive correlation for the Fermi hard source population (1.7 sigma). A correlation between 10-500 GeV flux and underdense fraction along the line of sight for VHE-like and Fermi hard sources is found at 2.4 sigma and 2.6 sigma, respectively. The preference for underdense sight lines is not displayed by gamma-ray emitting galaxies within the second Fermi catalog, containing sources detected above 100 MeV, or the SDSS DR7 quasar catalog. We investigate whether this marginal correlation might be a result of lower extragalactic background light (EBL) photon density within the underdense regions and find that, even in the most extreme case of a entirely underdense sight line, the EBL photon density is only 2% less than the nominal EBL density. Translating this into gamma-ray attenuation along the line of sight for a highly attenuated source with opacity tau(E,z) ~5, we estimate that the attentuation of gamma-rays decreases no more than 10%. This decrease, although non-neglible, is unable to account for the apparent hard source correlation with underdense lines of sight.