We investigate the linear radio size properties of the $mu$Jy populations of radio-selected active galactic nuclei (AGN) and star-forming galaxies (SFGs) using a multi-resolution catalog based on the original VLA-COSMOS 3,GHz 0farcs75 resolution mosaic and its convolved images (up to a resolution of 2farcs2). The final catalog contains 6,399 radio sources above a 3,GHz total flux density of $S_T>20$ $mu$Jy (median $<S_T>=37$ $mu$Jy), with redshift information (median $<z>=1.0$), and multi-wavelength classification as SFGs, radio-excess AGN (RX-AGN), or non-radio-excess AGN (NRX-AGN). RX-AGN are those whose radio emission exceeds the star formation rate derived by fitting the global spectral energy distribution. We derive the evolution with redshift and luminosity of the median linear sizes of each class of objects. We find that RX-AGN are compact, with median sizes of $sim$ 1-2 kpc and increasing with redshift, corresponding to an almost constant angular size of 0farcs25. NRX-AGN typically have radio sizes a factor of 2 larger than the RX-AGN. The median radio size of SFGs is about 5 kpc up to $zsim 0.7$, and it decreases beyond this redshift. Using luminosity-complete subsamples of objects, we separately investigate the effect of redshift and luminosity dependance. We compare the radio sizes of SFGs with those derived in the rest-frame far-infrared (FIR) and UV bands. We find that SFGs have comparable sizes (within 15%) in the radio and rest-frame FIR, while the sizes measured in the UV-band are systematically larger than the radio sizes.