A galaxys stellar mass-to-light ratio ($M_star/L$) is a useful tool for converting luminosity to stellar mass ($M_star$). However, the practical utility of $M_star/L$ inferred from stellar population synthesis (SPS) models is limited by mismatches between the real and assumed models for star formation history (SFH) and dust geometry, both of which vary within galaxies. Here, we measure spatial variations in $M_star/L$ and their dependence on color, star formation history, and dust across the disk of M31, using a map of $M^mathrm{CMD}_star$ derived from color-magnitude diagrams of resolved stars in the Panchromatic Hubble Andromeda Treasury (PHAT) survey. First, we find comparable scatter in $M_star/L$ for the optical and mid-IR, contrary to the common idea that $M_star/L$ is less variable in the IR. Second, we confirm that $M_star/L$ is correlated with color for both the optical and mid-IR and report color vs. $M_star/L$ relations (CMLRs) in M31 for filters used in the Sloan Digital Sky Survey (SDSS) and Widefield Infrared Survey Explorer (WISE). Third, we show that the CMLR residuals correlate with recent SFH, such that quiescent regions are offset to higher $M_star/L$ than star-forming regions at a fixed color. The mid-IR CMLR, however, is not linear due to the high scatter of $M_star/L$ in star-forming regions. Finally, we find a flatter optical CMLR than any SPS-based CMLRs in the literature. We show this is an effect of dust geometry, which is typically neglected but should be accounted for when using optical data to map $M_star/L$.