Isolated dense molecular cores are investigated to study the onset of complex organic molecule formation in interstellar ice. Sampling three cores with ongoing formation of low-mass stars (B59, B335, and L483) and one starless core (L694-2) we sample lines of sight to nine background stars and five young stellar objects (YSOs; A_K ~0.5 - 4.7). Spectra of these stars from 2-5 $mu$m with NASAs Infrared Telescope Facility (IRTF) simultaneously display signatures from the cores of H$_2$O (3.0 $mu$m), CH$_3$OH (C-H stretching mode, 3.53 $mu$m) and CO (4.67 $mu$m) ices. The CO ice is traced by nine stars in which five show a long wavelength wing due to a mixture of CO with polar ice (CO$_r$), presumably CH$_3$OH. Two of these sight lines also show independent detections of CH$_3$OH. For these we find the ratio of the CH$_3$OH:CO$_r$ is 0.55$pm$0.06 and 0.73$pm$0.07 from L483 and L694-2, respectively. The detections of both CO and CH$_3$OH for the first time through lines of sight toward background stars observationally constrains the conversion of CO into CH$_3$OH ice. Along the lines of sight most of the CO exists in the gas phase and $leq$15% of the CO is frozen out. However, CH$_3$OH ice is abundant with respect to CO (~50%) and exists mainly as a CH$_3$OH-rich CO ice layer. Only a small fraction of the lines of sight contains CH$_3$OH ice, presumably that with the highest density. The high conversion of CO to CH$_3$OH can explain the abundances of CH$_3$OH ice found in later stage Class 1 low mass YSO envelopes (CH$_3$OH:CO$_r$ ~ 0.5-0.6). For high mass YSOs and one Class 0 YSO this ratio varies significantly implying local variations can affect the ice formation. The large CH$_3$OH ice abundance indicates that the formation of complex organic molecules is likely during the pre-stellar phase in cold environments without higher energy particle interactions (e.g. cosmic rays).