Models of galaxy evolution assume some connection between the AGN and star formation activity in galaxies. We use the multi-wavelength information of the CDFS to assess this issue. We select the AGNs from the 3Ms XMM-Newton survey and measure the star-formation rates of their hosts using data that probe rest-frame wavelengths longward of 20 um. Star-formation rates are obtained from spectral energy distribution fits, identifying and subtracting an AGN component. We divide the star-formation rates by the stellar masses of the hosts to derive specific star-formation rates (sSFR) and find evidence for a positive correlation between the AGN activity (proxied by the X-ray luminosity) and the sSFR for the most active systems with X-ray luminosities exceeding Lx=10^43 erg/s and redshifts z~1. We do not find evidence for such a correlation for lower luminosity systems or those at lower redshifts. We do not find any correlation between the SFR (or the sSFR) and the X-ray absorption derived from high-quality XMM-Newton spectra either, showing that the absorption is likely to be linked to the nuclear region rather than the host, while the star-formation is not nuclear. Comparing the sSFR of the hosts to the characteristic sSFR of star-forming galaxies at the same redshift we find that the AGNs reside mostly in main-sequence and starburst hosts, reflecting the AGN - sSFR connection. Limiting our analysis to the highest X-ray luminosity AGNs (X-ray QSOs with Lx>10^44 erg/s), we find that the highest-redshift QSOs (with z>2) reside predominantly in starburst hosts, with an average sSFR more than double that of the main sequence, and we find a few cases of QSOs at z~1.5 with specific star-formation rates compatible with the main-sequence, or even in the quiescent region. (abridged)