We analyze the phenomenological consequences of embedding a flavor symmetry based on the groups $A_5$ and CP in a supersymmetric framework. We concentrate on the leptonic sector, where two different residual symmetries are assumed to be conserved at LO for charged and neutral leptons. All possible realizations to generate neutrino masses at tree level are investigated. Sizable flavor violating effects in the charged lepton sector are unavoidable due to the non-universality of soft-breaking terms determined by the symmetry. We derive testable predictions for the neutrino spectrum, lepton mixing and flavor changing processes with non-trivial relations among observables.