We took advantage of the observations carried out by XMM in the COSMOS field during 3.5 years, to study the long term variability of a large sample of AGN (638 sources), in a wide range of redshift (0.1<z<3.5) and X-ray luminosity ($10^{41}<$L(2-10)$<10^{45.5}$). Both a simple statistical method to asses the significance of variability, and the Normalized Excess Variance ($sigma^{2}_{rms}$) parameter, where used to obtain a quantitative measurement of the variability. Variability is found to be prevalent in most AGN, whenever we have good statistic to measure it, and no significant differences between type-1 and type-2 AGN were found. A flat (slope -0.23+/-0.03) anti-correlation between $sigma^{2}_{rms}$ and X-ray luminosity is found, when significantly variable sources are considered all together. When divided in three redshift bins, the anti-correlation becomes stronger and evolving with z, with higher redshift AGN being more variable. We prove however that this effect is due to the pre-selection of variable sources: considering all the sources with available $sigma^{2}_{rms}$ measurement, the evolution in redshift disappears. For the first time we were also able to study the long term X-ray variability as a function of $M_{rm BH}$ and Eddington ratio, for a large sample of AGN spanning a wide range of redshift. An anti-correlation between $sigma^{2}_{rms}$ and $M_{rm BH}$ is found, with the same slope of the anti-correlation between $sigma^{2}_{rms}$ and X-ray luminosity, suggesting that the latter can be a byproduct of the former one. No clear correlation is found between $sigma^{2}_{rms}$ and the Eddington ratio in our sample. Finally, no correlation is found between the X-ray $sigma^{2}_{rms}$ and the optical variability.