We analytically derive the upper bound on the overall efficiency of single-photon generation based on cavity quantum electrodynamics (QED), where cavity internal loss is treated explicitly. The internal loss leads to a tradeoff relation between the internal generation efficiency and the escape efficiency, which results in a fundamental limit on the overall efficiency. The corresponding lower bound on the failure probability is expressed only with an internal cooperativity, introduced here as the cooperativity parameter with respect to the cavity internal loss rate. The lower bound is obtained by optimizing the cavity external loss rate, which can be experimentally controlled by designing or tuning the transmissivity of the output coupler. The model used here is general enough to treat various cavity-QED effects, such as the Purcell effect, on-resonant or off-resonant cavity-enhanced Raman scattering, and vacuum-stimulated Raman adiabatic passage. A repumping process, where the atom is reused after its decay to the initial ground state, is also discussed.