In two-dimensional electron systems confined to wide AlAs quantum wells, composite fermions around the filling factor $ u$ = 3/2 are fully spin polarized but possess a valley degree of freedom. Here we measure the energy needed to completely valley polarize these composite fermions as a function of electron density. Comparing our results to the existing theory, we find overall good quantitative agreement, but there is an unexpected trend: The measured composite fermion valley polarization energy, normalized to the Coulomb energy, decreases with decreasing density.