We show that the rms-flux relation recently discovered in the X-ray light curves of Active Galactic Nuclei (AGN) and X-ray binaries (XRBs) implies that the light curves have a formally non-linear, exponential form, provided the rms-flux relation applies to variations on all time-scales (as it appears to). This phenomenological model implies that stationary data will have a lognormal flux distribution. We confirm this result using an observation of Cyg X-1, and further demonstrate that our model predicts the existence of the powerful millisecond flares observed in Cyg X-1 in the low/hard state, and explains the general shape and amplitude of the bicoherence spectrum in that source. Our model predicts that the most variable light curves will show the most extreme non-linearity. This result can naturally explain the apparent non-linear variability observed in some highly variable Narrow Line Seyfert 1 (NLS1) galaxies, as well as the low states observed on long time-scales in the NLS1 NGC 4051, as being nothing more than extreme manifestations of the same variability process that is observed in XRBs and less variable AGN. That variability process must be multiplicative (with variations coupled together on all time-scales) and cannot be additive (such as shot-noise), or related to self-organised criticality, or result from completely independent variations in many separate emitting regions. Successful models for variability must reproduce the observed rms-flux relation and non-linear behaviour, which are more fundamental characteristics of the variability process than the power spectrum or spectral-timing properties. Models where X-ray variability is driven by accretion rate variations produced at different radii remain the most promising.