We demonstrate an efficient experimental scheme for producing polarization-entangled photon pairs from spontaneous four-wave mixing (SFWM) in a laser-cooled $^{85}$Rb atomic ensemble, with a bandwidth (as low as 0.8 MHz) much narrower than the rubidium atomic natural linewidth. By stabilizing the relative phase between the two SFWM paths in a Mach-Zehnder interferometer configuration, we are able to produce all four Bell states. These subnatural-linewidth photon pairs with polarization entanglement are ideal quantum information carriers for connecting remote atomic quantum nodes via efficient light-matter interaction in a photon-atom quantum network.