Studying the temporal variability of BL Lac objects at the highest energies provides unique insights into the extreme physical processes occurring in relativistic jets and in the vicinity of super-massive black holes. To this end, the long-term variability of the BL Lac object PKS 2155-304 is analyzed in the high (HE, 100 MeV < E < 300 GeV) and very high energy (VHE, E > 200 GeV) gamma-ray domain. Over the course of ~9 yr of H.E.S.S observations the VHE light curve in the quiescent state is consistent with a log-normal behavior. The VHE variability in this state is well described by flicker noise (power-spectral-density index {ss}_VHE = 1.10 +0.10 -0.13) on time scales larger than one day. An analysis of 5.5 yr of HE Fermi LAT data gives consistent results ({ss}_HE = 1.20 +0.21 -0.23, on time scales larger than 10 days) compatible with the VHE findings. The HE and VHE power spectral densities show a scale invariance across the probed time ranges. A direct linear correlation between the VHE and HE fluxes could neither be excluded nor firmly established. These long-term-variability properties are discussed and compared to the red noise behavior ({ss} ~ 2) seen on shorter time scales during VHE-flaring states. The difference in power spectral noise behavior at VHE energies during quiescent and flaring states provides evidence that these states are influenced by different physical processes, while the compatibility of the HE and VHE long-term results is suggestive of a common physical link as it might be introduced by an underlying jet-disk connection.