We study the contributions of supersymmetric models with a $U(1)$ horizontal symmetry and only spontaneous CP breaking to various lepton flavor observables, such as $mu to egamma$ and the electron electric dipole moment. We show that both a horizontal symmetry and a lack of explicit CP violation can alleviate the existing bounds from such observables. The undetermined $mathcal{O}(1)$ coefficients in such mass matrix models muddle the interpretation of the bounds from various flavor observables. To overcome this, we define a new fine-tuning measure for different observables in such setups. This allows us to study how naturally the observed IR flavor observables can emerge from a given mass matrix model. We use our flavor-naturalness measure in study of our supersymmetric models and quantify the degree of fine tuning required by the bounds from various lepton flavor observables at each mass scale of sleptons, neutralinos, and charginos.