We present a quantum repeater protocol using atomic ensembles, linear optics and single-photon sources. Two local polarization entangled states of atomic ensembles $u$ and $d$ are generated by absorbing a single photon emitted by an on-demand single-photon sources, based on which high-fidelity local entanglement between four ensembles can be established efficiently through Bell-state measurement. Entanglement in basic links and entanglement connection between links are carried out by the use of two-photon interference. In addition to being robust against phase fluctuations in the quantum channels, this scheme may speed up quantum communication with higher fidelity by about 2 orders of magnitude for 1280 km compared with the partial read (PR) protocol (Sangouard {it et al.}, Phys. Rev. A {bf77}, 062301 (2008)) which may generate entanglement most quickly among the previous schemes with the same ingredients.