We have used coherent, resonant, x-ray magnetic speckle patterns to measure the statistical evolution of the microscopic magnetic domains in perpendicular magnetic films as a function of the applied magnetic field. Our work constitutes the first direct, ensemble-averaged study of microscopic magnetic return point memory, and demonstrates the profound impact of interfacial roughness on this phenomenon. At low fields, the microscopic magnetic domains forget their past history with an exponential field dependence.