The unprecedented brilliance of X-ray free-electron lasers (XFELs) [1, 2] has enabled first studies of nonlinear interactions in the hard X-ray range. In particular, X-ray-optical mixing [3], X-ray second harmonic generation (XSHG) [4] and nonlinear Compton scattering (NLCS) [5] have been recently observed for the first time using XFELs. The former two experiments as well as X-ray parametric downconversion (XPDC)[6, 7] are well explained by nonlinearities in the impulse approximation[8], where electrons in a solid target are assumed to be quasi free for X-ray interactions far from atomic resonances. However, the energy of the photons generated in NLCS at intensities reaching up to 4 x 1020 W/cm2 exhibit an anomalous red-shift that is in violation with the free-electron model. Here we investigate the underlying physics of X-ray nonlinear interactions at intensities on order of 1016 W/cm2. Specifically, we perform a systematic study of XSHG in diamond. While one phase-matching geometry has been measured in Shwartz et al.[4], we extend these studies to multiple Fourier components and with significantly higher statistics, which allows us to determine the second order nonlinear structure factor. We measure the efficiency, angular dependence, and contributions from different source terms of the process. We find good agreement of our measurements with the quasi-free electron model.