We present a general framework for cavity quantum electrodynamics with strongly frequency-dependent mirrors. The method is applicable to a variety of reflectors exhibiting sharp internal resonances as can be realized, for example, with photonic-crystal mirrors or with two-dimensional atomic arrays around subradiant points. Our approach is based on a modification of the standard input--output formalism to explicitly include the dynamics of the mirrors internal resonance. We show how to directly extract the interaction tuning parameters from the comparison with classical transfer matrix theory and how to treat the non-Markovian dynamics of the cavity field mode introduced by the mirrors internal resonance. As an application within optomechanics, we illustrate how a non-Markovian Fano cavity possessing a flexible photonic crystal mirror can provide both sideband resolution as well as strong heating suppression in optomechanical cooling. This approach, amenable to a wide range of systems, opens up possibilities for using hybrid frequency-dependent reflectors in cavity quantum electrodynamics for engineering novel forms of light-matter interactions.