We study the local disorder in the deformation of amorphous materials by decomposing the particle displacements into a continuous, inhomogeneous field and the corresponding fluctuations. We compare these fields to the commonly used non-affine displacements in an elastically deformed 2D Lennard-Jones glass. Unlike the non-affine field, the fluctuations are very localized, and exhibit a much smaller (and system size independent) correlation length, on the order of a particle diameter, supporting the applicability of the notion of local defects to such materials. We propose a scalar noise field to characterize the fluctuations, as an additional field for extended continuum models, e.g., to describe the localized irreversible events observed during plastic deformation.