We have measured de Haas-van Alphen oscillations of Cr$_{1-x}$V$_x$, $0 le x le 0.05$, at high fields for samples on both sides of the quantum critical point at $x_c=0.035$. For all samples we observe only those oscillations associated with a single small hole band with magnetic breakdown orbits of the reconstructed Fermi surface evident for $x<x_c$. The absence of oscillations from Fermi surface sheets most responsible for the spin density wave (SDW) in Cr for $x>x_c$ is further evidence for strong fluctuation scattering of these charge carriers well into the paramagnetic regime. We find no significant mass enhancement of the carriers in the single observed band at any $x$. An anomalous field dependence of the dHvA signal for our $x=0.035$ crystal at particular orientations of the magnetic field is identified as due to magnetic breakdown that we speculate results from a field induced SDW transition at high fields.