Although extensively investigated, the role of the environment in galaxy formation is still not well understood. In this context, the Galaxy Stellar Mass Function (GSMF) is a powerful tool to understand how environment relates to galaxy mass assembly and the quenching of star-formation. In this work, we make use of the high-precision photometric redshifts of the UltraVISTA Survey to study the GSMF in different environments up to $z sim 3$, on physical scales from 0.3 to 2 Mpc, down to masses of $M sim 10^{10} M_{odot}$. We witness the appearance of environmental signatures for both quiescent and star-forming galaxies. We find that the shape of the GSMF of quiescent galaxies is different in high- and low-density environments up to $z sim 2$ with the high-mass end ($M gtrsim 10^{11} M_{odot}$) being enhanced in high-density environments. On the contrary, for star-forming galaxies a difference between the GSMF in high- and low density environments is present for masses $M lesssim 10^{11} M_{odot}$. Star-forming galaxies in this mass range appear to be more frequent in low-density environments up to $z < 1.5$. Differences in the shape of the GSMF are not visible anymore at $z > 2$. Our results, in terms of general trends in the shape of the GSMF, are in agreement with a scenario in which galaxies are quenched when they enter hot gas-dominated massive haloes which are preferentially in high-density environments.