The Pierre Auger Collaboration has reported evidence for anisotropy in the distribution of arrival directions of the cosmic rays with energies $E>E_{th}=5.5times 10^{19}$ eV. These show a correlation with the distribution of nearby extragalactic objects, including an apparent excess around the direction of Centaurus A. If the particles responsible for these excesses at $E>E_{th}$ are heavy nuclei with charge $Z$, the proton component of the sources should lead to excesses in the same regions at energies $E/Z$. We here report the lack of anisotropies in these directions at energies above $E_{th}/Z$ (for illustrative values of $Z=6, 13, 26$). If the anisotropies above $E_{th}$ are due to nuclei with charge $Z$, and under reasonable assumptions about the acceleration process, these observations imply stringent constraints on the allowed proton fraction at the lower energies.