We derive the CO-to-H2 conversion factor, X_CO = N(H2)/I_CO, across the Perseus molecular cloud on sub-parsec scales by combining the dust-based N(H2) data with the I_CO data from the COMPLETE Survey. We estimate an average X_CO ~ 3 x 10^19 cm^-2 K^-1 km^-1 s and find a factor of ~3 variations in X_CO between the five sub-regions in Perseus. Within the individual regions, X_CO varies by a factor of ~100, suggesting that X_CO strongly depends on local conditions in the interstellar medium. We find that X_CO sharply decreases at Av < 3 mag but gradually increases at Av > 3 mag, with the transition occurring at Av where I_CO becomes optically thick. We compare the N(HI), N(H2), I_CO, and X_CO distributions with two models of the formation of molecular gas, a one-dimensional photodissociation region (PDR) model and a three-dimensional magnetohydrodynamic (MHD) model tracking both the dynamical and chemical evolution of gas. The PDR model based on the steady state and equilibrium chemistry reproduces our data very well but requires a diffuse halo to match the observed N(HI) and I_CO distributions. The MHD model generally matches our data well, suggesting that time-dependent effects on H2 and CO formation are insignificant for an evolved molecular cloud like Perseus. However, we find interesting discrepancies, including a broader range of N(HI), likely underestimated I_CO, and a large scatter of I_CO at small Av. These discrepancies likely result from strong compressions/rarefactions and density fluctuations in the MHD model.