The search for a possible critical point in the QCD phase diagram is ongoing in heavy ion collision experiments at RHIC which scan the phase diagram by scanning the beam energy; a coming upgrade will increase the luminosity and extend the rapidity acceptance of the STAR detector. In fireballs produced in RHIC collisions, the baryon density depends on rapidity. By employing Ising universality together with a phenomenologically motivated freezeout prescription, we show that the resulting rapidity dependence of cumulant observables sensitive to critical fluctuations is distinctive. The dependence of the kurtosis (of the event-by-event distribution of the number of protons) on rapidity near mid-rapidity will change qualitatively if a critical point is passed in the scan. Hence, measuring the rapidity dependence of cumulant observables can enhance the prospect of discovering a critical point, in particular if it lies between two energies in the beam energy scan.