In dark matter direct detection experiments, neutron is a serious source of background, which can mimic the dark matter-nucleus scattering signals. In this paper, we present an improved evaluation of the neutron background in the PandaX-II dark matter experiment by a novel approach. Instead of fully relying on the Monte Carlo simulation, the overall neutron background is determined from the neutron-induced high energy signals in the data. In addition, the probability of producing a dark-matter-like background per neutron is evaluated with a complete Monte Carlo generator, where the correlated emission of neutron(s) and $gamma$(s) in the ($alpha$, n) reactions and spontaneous fissions is taken into consideration. With this method, the neutron backgrounds in the Run 9 (26-ton-day) and Run 10 (28-ton-day) data sets of PandaX-II are estimated to be 0.66$pm$0.24 and 0.47$pm$0.25 events, respectively.