By including the interference effect between the QCD and the QED diagrams, we carry out a complete analysis on the exclusive productions of $e^+e^- to J/psi+chi_{cJ}$ ($J=0,1,2$) at the $B$ factories with $sqrt{s}=10.6$ GeV at the next-to-leading-order (NLO) level in $alpha_s$, within the nonrelativistic QCD framework. It is found that the $mathcal O (alpha^3alpha_s)$-order terms that represent the tree-level interference are comparable with the usual NLO QCD corrections, especially for the $chi_{c1}$ and $chi_{c2}$ cases. To explore the effect of the higher-order terms, namely $mathcal O (alpha^3alpha_s^2)$, we perform the QCD corrections to these $mathcal O (alpha^3alpha_s)$-order terms for the first time, which are found to be able to significantly influence the $mathcal O (alpha^3alpha_s)$-order results. In particular, in the case of $chi_{c1}$ and $chi_{c2}$, the newly calculated $mathcal O (alpha^3alpha_s^2)$-order terms can to a large extent counteract the $mathcal O (alpha^3alpha_s)$ contributions, evidently indicating the indispensability of the corrections. In addition, we find that, as the collision energy rises, the percentage of the interference effect in the total cross section will increase rapidly, especially for the $chi_{c1}$ case.