We introduce a filter using a noise-free quantum buffer with large optical bandwidth that can both filter temporal-spectral modes, as well as inter-convert them and change their frequency. We show that such quantum buffers optimally filter out temporal-spectral noise; producing identical single-photons from many distinguishable noisy single-photon sources with the minimum required reduction in brightness. We then experimentally demonstrate a noise-free quantum buffer in a warm atomic system that is well matched to quantum dots and can outperform all intensity (incoherent) filtering schemes for increasing indistinguishability.