N-doped single/multi-walled carbon nanotubes (CNTs) were studied for long time from synthesis to properties. However, the stability of N in the CNT lattice still needs further developments. In this work, to obtain more stable N-doped CNTs, concentric double-walled (DW) CNTs with more N were synthesized using benzylamine as C and N source. In order to test the stability of N-doped DWCNTs, high-temperature annealing in vacuum was performed. By XPS and Raman spectroscopic measurements, we found that the N-doped DWCNTs are still stable under 1500 $,^{circ}mathrm{C}$: the graphitic N does not change at all, the molecular N is partly removed, and the pyridinic N ratio greatly increases by more than two times. The reason could be that the N atoms from the surrounded N-contained materials combine into the CNT lattice during the annealing. Compared with the undoped DWCNTs, no Raman frequency shift was observed for the RBM, the G-band, and the G-band of the N-doped DWCNTs.