We present the systematic analysis of the UVOT and XRT light curves for a sample of 26 Swift Gamma-Ray Bursts (GRBs). By comparing the optical/UV and X-ray light curves, we found that they are remarkably different during the first 500s after the BAT trigger, while they become more similar during the middle phase of the afterglow, i.e. between 2000s and 20000s. If we take literally the average properties of the sample, we find that the mean temporal indices observed in the optical/UV and X-rays after 500s are consistent with a forward-shock scenario, under the assumptions that electrons are in the slow cooling regime, the external medium is of constant density and the synchrotron cooling frequency is situated between the optical/UV and X-ray observing bands. While this scenario describes well the averaged observed properties, some individual GRB afterglows require different or additional assumptions, such as the presence of late energy injection. We show that a chromatic break (a break in the X-ray light curve that is not seen in the optical) is present in the afterglows of 3 GRBs and demonstrate evidence for chromatic breaks in a further 4 GRBs. The average properties of these breaks cannot be explained in terms of the passage of the synchrotron cooling frequency through the observed bands, nor a simple change in the external density. It is difficult to reconcile chromatic breaks in terms of a single component outflow and instead, more complex jet structure or additional emission components are required.