We present dust attenuation properties of spectroscopically confirmed star forming galaxies on the main sequence at a redshift of ~4.4-5.8. Our analyses are based on the far infrared continuum observations of 118 galaxies at rest-frame 158{mu}m obtained with the Atacama Large Millimeter Array (ALMA) Large Program to INvestigate [CII] at Early times (ALPINE). We study the connection between the ultraviolet (UV) spectral slope ($beta$), stellar mass (M_*), and infrared excess (IRX=L_IR/L_UV). Twenty-three galaxies are individually detected in the continuum at >3.5 sigma significance. We perform a stacking analysis using both detections and nondetections to study the average dust attenuation properties at z~4.4-5.8. The individual detections and stacks show that the IRX-$beta$ relation at z~5 is consistent with a steeper dust attenuation curve than typically found at lower redshifts (z<4). The attenuation curve is similar to or even steeper than that of the extinction curve of the Small Magellanic Cloud (SMC). This systematic change of the IRX-$beta$ relation as a function of redshift suggests an evolution of dust attenuation properties at z>4. Similarly, we find that our galaxies have lower IRX values, up to 1dex on average, at a fixed mass compared to previously studied IRX-M_* relations at z<4, albeit with significant scatter. This implies a lower obscured fraction of star formation than at lower redshifts. Our results suggest that dust properties of UV-selected star forming galaxies at z>4 are characterised by (i) a steeper attenuation curve than at z<4, and (ii) a rapidly decreasing dust obscured fraction of star formation as a function of redshift. Nevertheless, even among this UV-selected sample, massive galaxies (log M_*/$M_odot$>10) at z~5-6 already exhibit an obscured fraction of star formation of ~45%, indicating a rapid build-up of dust during the epoch of reionization.