Maximizing the utility of upcoming photometric cluster surveys requires a thorough understanding of the richness-mass relation of galaxy clusters. We use Monte Carlo simulations to study the impact of various sources of observational scatter on this relation. Cluster ellipticity, photometric errors, photometric redshift errors, and cluster-to-cluster variations in the properties of red-sequence galaxies contribute negligible noise. Miscentering, however, can be important, and likely contributes to the scatter in the richness-mass relation of galaxy maxBCG clusters at the low mass end, where centering is more difficult. We also investigate the impact of projection effects under several empirically motivated assumptions about cluster environments. Using SDSS data and the maxBCG cluster catalog, we demonstrate that variations in cluster environments can rarely (approx 1% - 5% of the time) result in significant richness boosts. Due to the steepness of the mass/richness function, the corresponding fraction of optically selected clusters that suffer from these projection effects is approx 5% - 15%. We expect these numbers to be generic in magnitude, but a precise determination requires detailed, survey-specific modeling.