Flat-band systems are a promising platform for realizing exotic collective ground states with spontaneously broken symmetry because the electron-electron interactions dominate the kinetic energy. A state of particular interest would be the chased after interlayer-phase-coherent exciton condensate but the conventional treatments of the effect of thermal and quantum fluctuations suggest that this state might be either unstable or fragile. In this work, using double twisted bilayer graphene heterostructures as an example, we show that the quantum metric of the Bloch wave functions can strongly stabilize the exciton condensate and reverse the conclusion that one would draw using a conventional approach. The quantum metric contribution arises from interband terms, and flat-bands are most commonly realized by engineering multiband systems. Our results therefore suggest that in many practical situations the quantum metric can play a critical role in determining the stability of exciton condensates in double layers formed by two-dimensional systems with flat-bands.