The existence of dark radiation that is completely decoupled from the standard model in the early Universe leaves open the possibility of an associated dark radiation isocurvature mode. We show that the presence of dark radiation isocurvature leads to spatial variation in the primordial abundances of helium and deuterium due to spatial variation in $N_{rm eff}$ during Big Bang nucleosynthesis. We use the result to constrain the existence of such an isocurvature mode on scales down to $sim 1$ Mpc scales. By measuring the excess variance in the primordial helium to hydrogen and deuterium to hydrogen ratio in different galaxies, we constrain the variance in average isocurvature in a galaxy to be less than $0.13/Delta bar{N}_{rm eff}$ at 95% confidence. Here $Delta bar{N}_{rm eff}$ is the spatially averaged increase in $N_{rm eff}$ due to the additional dark radiation component.