It is often assumed that atoms are hard spheres in the estimation of local lattice distortion (LLD) in high-entropy alloys (HEAs). However, our study demonstrates that the hard sphere model misses the key effect, charge transfer among atoms with different electronegativities, in the understanding of the stabilization of severely-distorted HEAs. Through the characterization and simulations of the local structure of the HfNbTiZr HEA, we found that the charge transfer effect competes with LLD to significantly reduce the average atomic-size mismatch. Our finding may form the basis for the design of severely distorted, but stable HEAs.