Simultaneous measurement of both neutrons and charged particles emitted in the reaction $^{64}$Zn + $^{64}$Zn at 45 MeV/nucleon allows comparison of the neutron to proton ratio at midrapidity with that at projectile rapidity. The evolution of N/Z in both rapidity regimes with increasing centrality is examined. For the completely re-constructed midrapidity material one finds that the neutron-to-proton ratio is above that of the overall $^{64}$Zn + $^{64}$Zn system. In contrast, the re-constructed ratio for the quasiprojectile is below that of the overall system. This difference provides the most complete evidence to date of neutron enrichment of midrapidity nuclear matter at the expense of the quasiprojectile.