We study the long term evolution of the relationship between the root mean square (rms) variability and flux (the ``rms-flux relation) for the black hole Cygnus X-1 as monitored from 1996 to 2003 with the Rossi X-ray Timing Explorer (RXTE). We confirm earlier results by Uttley & McHardy (2001) of a linear relationship between rms and flux in the hard state on time scales > 5 s reflecting in its slope the fractional rms variability. We demonstrate the perpetuation of the linear rms-flux relation in the soft and the intermediate state. The existence of a non-zero intercept in the linear rms-flux relation argues for two lightcurve components, for example, one variable and one non-variable component, or a possible constant rms component. The relationship between these two hypothesized components can be described by a fundamental dependence of slope and intercept at time scales ~< 10 ksec with long term averages of the flux.