We present the first measurement of the inclusive three-jet differential cross section as a function of the invariant mass of the three jets with the largest transverse momenta in an event in p anti-p collisions at sqrt(s) = 1.96 TeV. The measurement is made in different rapidity regions and for different jet transverse momentum requirements and is based on a data set corresponding to an integrated luminosity of 0.7 fb^{-1} collected with the D0 detector at the Fermilab Tevatron Collider. The results are used to test the three-jet matrix elements in perturbative QCD calculations at next-to-leading order in the strong coupling constant. The data allow discrimination between parametrizations of the parton distribution functions of the proton.