An important part of a CFT four-point function, the stress tensor sector, comprises the exchanges of the stress tensor and its composites. The OPE coefficients of these multi-stress tensor operators and consequently, the complete stress tensor sector of four-point functions in CFTs with a large central charge, can be determined by computing a heavy-heavy-light-light correlator. We show how one can make substantial progress in this direction by bootstrapping a certain ansatz for the stress tensor sector of the correlator, iteratively computing the OPE coefficients of multi-stress tensor operators with increasing twist. Some parameters are not fixed by the bootstrap - they correspond to the OPE coefficients of multi-stress tensors with spin zero and two. We further show that in holographic CFTs one can use the phase shift computed in the dual gravitational theory to reduce the set of undetermined parameters to the OPE coefficients of multi-stress tensors with spin zero. Finally, we verify some of these results using the Lorentzian OPE inversion formula and comment on its regime of applicability.