We outline a machine learning strategy for determining the effective interaction in the condensed phases of matter using scattering. Via a case study of colloidal suspensions, we showed that the effective potential can be probabilistically inferred from the scattering spectra without any restriction imposed by model assumptions. Comparisons to existing parametric approaches demonstrate the superior performance of this method in accuracy, efficiency, and applicability. This method can effectively enable quantification of interaction in highly correlated systems using scattering and diffraction experiments.