Graphene is an ideal material for hot-electron bolometers, due to its low heat capacity and weak electron-phonon coupling. Nanostructuring graphene with quantum dot constrictions yields detectors with extraordinarily high intrinsic responsivity, higher than 1x10^9 V/W at 3K. The sensing mechanism is bolometric in nature: the quantum confinement gap causes a strong dependence of the electrical resistance on the electron temperature. Here we show that this quantum confinement gap does not impose a limitation on the photon energy for light detection and these quantum dot bolometers work in a very broad spectral range, from terahertz, through telecom to ultraviolet radiation, with responsivity independent of wavelength. We also measure the power dependence of the response. Although the responsivity decreases with increasing power, it stays higher than 1x10^8 V/W in a wide range of absorbed power, from 1 pW to 0.4 nW.