We study time evolution of critical fluctuations of conserved charges near the QCD critical point in the context of relativistic heavy ion collisions. A stochastic diffusion equation is employed in order to describe the diffusion property of the critical fluctuation arising from the coupling of the order parameter field to conserved charges. We show that the diffusion property gives rise to a possibility of probing the early time fluctuations through the rapidity window dependence of the second-order cumulant and correlation function of conserved charges. It is pointed out that their non-monotonic behaviors as functions of the rapidity interval are robust experimental signals for the existence of the critical enhancement around the QCD critical point.