Defect microstructures formed in ion-irradiated metals, for example iron or tungsten, often exhibit patterns of spatially ordered nano-scale dislocation loops. We show that such ordered dislocation loop structures may form spontaneously as a result of Brownian motion of loops, biased by the angular-dependent elastic interaction between the loops. Patterns of spatially ordered loops form once the local density of loops produced by ion irradiation exceeds a critical threshold value.