We present a galaxy catalog simulator which turns N-body simulations with subhalos into multiband photometric mocks. The simulator assigns galaxy properties to each subhalo to reproduce the observed cluster galaxy halo occupation distribution, the radial and mass dependent variation in fractions of blue galaxies, the luminosity functions in clusters and the field, and the red-sequence in clusters. Moreover, the evolution of these parameters is tuned to match existing observational constraints. Field galaxies are sampled from existing multiband photometric surveys using derived galaxy photometric redshifts. Parametrizing an ensemble of cluster galaxy properties enables us to create mock catalogs with variations in those properties, which in turn allows us to quantify the sensitivity of cluster finding to current observational uncertainties in these properties. We present an application of the catalog simulator to characterize the selection function of a galaxy cluster finder that utilizes the cluster red-sequence galaxy clustering on the sky, in terms of completeness and contamination. We estimate systematic uncertainties due to the observational uncertainties on our simulator parameters in determining the selection function using five different sets of modified catalogs. Our estimates indicate that these uncertainties are at the $le15$% level with current observational constraints on cluster galaxy populations and their evolution. In addition, we examine the $B_{gc}$ parameter as an optical mass indicator and measure the intrinsic scatter of the $B_{gc}$--mass relation to be approximately log normal with $sigma_{log_{10}M}sim0.25$. Finally, we present tests of a red sequence overdensity redshift estimator using both simulated and real data, showing that it delivers redshifts for massive clusters with $sim$2% accuracy out to redshifts $zsim0.5$ with SDSS-like datasets.