We show that in the Standard Model the parametrically leading (by a factor 1/alpha_s) contribution to the inclusive CP asymmetry in B->X_{s,d}+gamma decays arises from a long-distance effect in the interference of the electromagnetic dipole amplitude with the amplitude for an up-quark penguin transition accompanied by soft gluon emission. This contribution is governed by a single hadronic parameter Lambda_{17}^u related to a matrix elements of a non-local operator. In view of current experimental data, a future precision measurement of the flavor-averaged CP asymmetry in B->X_s+gamma will signal the presence of new physics only if a value below -2% is found. A cleaner probe of new physics is offered by the difference of the CP asymmetries in charged versus neutral B-meson decays.