We fabricate van der Waals heterostructure devices using few unit cell thick Bi$_2$Sr$_2$CaCu$_2$O$_{8+delta}$ for magnetotransport measurements. The superconducting transition temperature and carrier density in atomically thin samples can be maintained to close to that of the bulk samples. As in the bulk sample, the sign of the Hall conductivity is found to be opposite to the normal state near the transition temperature but with a drastic enlargement of the region of Hall sign reversal in the temperature-magnetic field phase diagram as the thickness of samples decreases. Quantitative analysis of the Hall sign reversal based on the excess charge density in the vortex core and superconducting fluctuations suggests a renormalized superconducting gap in atomically thin samples at the 2-dimensional limit.