We study the role of electron-phonon scattering for a pulse-triggered quantum dot single-photon source which utilizes a modified version of stimulated Raman adiabatic passage and cavity-coupling. This on-demand source is coherently pumped with an optical pulse in the presence of a continuous wave laser drive, allowing for efficient generation of indistinguishable single photons with polarizations orthogonal to the applied fields. In contrast to previous studies, we explore the role of electron-phonon scattering on this semiconductor system by using a polaron master equation approach to model the biexciton-exciton cascade and cavity mode coupling. In addition to background zero-phonon-line decoherence processes, electron-acoustic-phonon coupling, which usually degrades the indistinguishability and efficiency of semiconductor photon sources, is rigorously taken into account. We study how cavity and laser detunings affect the device performance, and explore the effects of finite temperature on pure dephasing and intrinsic phonon-coupling. We describe how this biexciton-exciton cascade scheme allows for true single photons to be generated with over 90% quantum indistinguishability and efficiency simultaneously using realistic experimental parameters. We also show how the double-field dressing can be probed through the cavity-emitted spectrum.