We investigate a $U(1)_{B-L}$ gauge extension of the Standard Model (SM) where the gauge boson mass is generated by the Stueckelberg mechanism. Three right-handed neutrinos are added to cancel the gauge anomaly and hence the neutrino masses can be explained. A new Dirac fermion could be a WIMP dark matter whose interaction with the SM sector is mediated by the new gauge boson. Assuming the perturbativity of the gauge coupling up to the Planck scale, we find that only the resonance region is feasible for the dark matter abundance. After applying the $Delta N_{eff}$ constraints from the current Planck experiment, the collider search constraints as well as the dark matter direct detection limits, we observe that the $B-L$ charge of dark matter satisfies $|Q_{chi}|>0.11$. Such a scenario might be probed conclusively by the projected CMB-S4 experiment, assuming the right-handed neutrinos are thermalized with the SM sector in the early universe.