We calculate the momentum dependence of the $rho^0-omega$ mixing amplitude in vacuum with vector nucleon-nucleon interaction in presence of a constant homogeneous weak magnetic field background. The mixing amplitude is generated by the nucleon-nucleon ($NN$) interaction and thus driven by the neutron-proton mass difference along with a constant magnetic field. We find a significant effect of magnetic field on the mixing amplitude. We also calculate the Charge symmetry violating (CSV) $NN$ potential induced by the magnetic field dependent mixing amplitude. The presence of the magnetic field influences the $NN$ potential substantially which can have important consequences in highly magnetized astrophysical compact objects, such as magnetars. The most important observation of this work is that the mixing amplitude is non-zero, leading to positive contribute to the CSV potential if the proton and neutron masses are taken to be equal.