A primary aim of the Nuclear Spectroscopic Telescope Array (NuSTAR) mission is to find and characterize heavily obscured Active Galactic Nuclei (AGNs). Based on mid-infrared photometry from the Wide-Field Infrared Survey Explorer (WISE) and optical photometry from the Sloan Digital Sky Surveys, we have selected a large population of luminous obscured AGN (i.e., obscured quasars). Here we report NuSTAR observations of four WISE-selected heavily obscured quasars for which we have optical spectroscopy from the Southern African Large Telescope and W. M. Keck Observatory. Optical diagnostics confirm that all four targets are AGNs. With NuSTAR hard X-ray observations, three of the four objects are undetected, while the fourth has a marginal detection. We confirm that these objects have observed hard X-ray (10-40 keV) luminosities at or below ~ 10^43 erg s^-1. We compare X-ray and IR luminosities to obtain estimates of the hydrogen column densities (N_H) based on the suppression of the hard X-ray emission. We estimate N_H of these quasars to be at or larger than 10^25 cm^-2, confirming that WISE and optical selection can identify very heavily obscured quasars that may be missed in X-ray surveys, and do not contribute significantly to the cosmic X-ray background. From the optical Balmer decrements, we found that our three extreme obscured targets lie in highly reddened host environments. This galactic extinction is not adequate to explain the more obscured AGN, but it may imply a different scale of obscuration in the galaxy.