We propose a simple mechanism for stabilizing flavon fields with aligned vacuum structure in models with discrete flavor symmetry. The basic idea is that flavons are stabilized by the balance between the negative soft mass and non-renormalizable terms in the potential. We explicitly discuss how our mechanism works in $A_4$ flavor model, and show that the field content is significantly simplified. It also works as a natural solution to the cosmological domain wall problem.