We present the detailed characterisation of a sample of 56 sources serendipitously detected in ALMA band 7, as part of the ALMA Large Program to INvestigate CII at Early Times (ALPINE) in COSMOS and ECDFS. These sources have been used to derive the total infrared luminosity function (LF) and to estimate the cosmic star formation rate density (SFRD) up to z=6. We have looked for counterparts in all the available multi-wavelength and photometric redshift catalogues, and in deeper near- and mid-IR source lists and maps, to identify optically dark sources with no matches in the public catalogues. Our ALMA blind survey allows us to push further the study of the nature and evolution of dusty galaxies at high-z, identifying luminous and massive sources to redshifts and faint luminosities never probed before by any far-infrared surveys. The ALPINE data are the first ones to sample the faint-end of the infrared LF, showing little evolution from z=2.5 to z=6, and a flat slope up to the highest redshifts. The SFRD obtained by integrating the luminosity function remains almost constant between z=2 and 6, and significantly higher than the optical/UV derivations, showing an important contribution of dusty galaxies and obscured star formation up to high-z. About 14 per cent of the ALPINE serendipitous continuum sources are optically+near-IR dark (six show a counterpart only in the mid-IR and no HST or near-IR identification, while two are detected as [CII] emitters at z=5). The six HST and near-IR dark galaxies with mid-IR counterpart contribute for about 17 per cent of the total SFRD at z=5 and dominate the high-mass end of the stellar mass function at z>3.