We study the co-evolution between the black hole accretion rate (BHAR) and the star formation rate (SFR) in different galaxy life phases: main sequence star-forming galaxies, quiescent and starburst galaxies at different cosmic epochs. We take advantage of the X-ray data from the Chandra COSMOS-Legacy survey and of the extensive multiwavelength ancillary observations in the COSMOS field presented in the COSMOS2015 catalog. We perform an X-ray stacking analysis and combine it with detected sources, in a broad redshift interval ($0.1<z<3.5$). The X-ray luminosity is used to predict the BHAR, while a similar stacking analysis on far-infrared Herschel maps is used to measure the corresponding SFR. We focus on the evolution of the average SFR-stellar mass (M*) relation and compare it with the BHAR-M* relation. We find that the ratio between BHAR and SFR does not evolve with redshift, although it depends on stellar mass. For the star-forming populations, this dependence on M* has a logarithmic slope of $sim0.6$, for the starburst sample of $sim0.4$, both at odds with quiescent sources where it remains constant ($log(rm {BHAR}/{rm SFR})sim -3.4$). By studying the specific BHAR and specific SFR we find signs of downsizing for both M* and black hole mass (M$_{rm BH}$): quiescents grew their super-massive black hole at very early times, while star-forming and starburst galaxies had an accretion that endured until more recent times. Our results support the idea that the same physical processes feed and sustain both star formation and black hole accretion. Our integrated estimates of the M*-M$_{rm BH}$ relation at all redshifts are consistent with independent determinations of the local M*-M$_{rm BH}$ relation, thus adding key evidence to a weak evolution in the BHAR/SFR, and its low normalization compared to local dynamical M*-M$_{rm BH}$ relations.