Atomic masses of the neutron-rich isotopes $^{76-80}$Zn, $^{78-83}$Ga, $^{80-85}Ge, $^{81-87}$As and $^{84-89}$Se have been measured with high precision using the Penning trap mass spectrometer JYFLTRAP at the IGISOL facility. The masses of $^{82,83}$Ga, $^{83-85}$Ge, $^{84-87}$As and $^{89}$Se were measured for the first time. These new data represent a major improvement in the knowledge of the masses in this neutron-rich region. Two-neutron separation energies provide evidence for the reduction of the N=50 shell gap energy towards germanium Z=32 and a subsequent increase at gallium (Z=31). The data are compared with a number of theoretical models. An indication of the persistent rigidity of the shell gap towards nickel (Z=28) is obtained.