Discrete flavor symmetry is often introduced for explaining quark/lepton masses and mixings. However, its spontaneous breaking leads to the appearance of domain walls, which is problematic for cosmology. We consider a possibility that the discrete flavor symmetry is anomalous under the color SU(3) so that it splits the energy levels of degenerate discrete vacua as a solution to the domain wall problem. We find that in most known models of flavor symmetry, the QCD anomaly effect can only partially remove the degeneracy and there still remain degenerate vacua.