A number of experimental measurements of the Casimir force have observed a logarithmic distance variation of the voltage that minimizes electrostatic force between the plates in a sphere-plane geometry. We show that this variation can be simply understood from a geometric averaging of surface potential patches together with the Proximity Force Approximation.