Bragg diffraction of an atomic wave packet in a retroreflective geometry with two counterpropagating optical lattices exhibits a light shift induced phase. We show that the temporal shape of the light pulse determines the behavior of this phase shift: In contrast to Raman diffraction, Bragg diffraction with Gaussian pulses leads to a significant suppression of the intrinsic phase shift due to a scaling with the third power of the inverse Doppler frequency. However, for box-shaped laser pulses, the corresponding shift is twice as large as for Raman diffraction. Our results are based on approximate, but analytical expressions as well as a numerical integration of the corresponding Schrodinger equation.