In analog to counterparts widely used in electronic circuits, all optical non-reciprocal devices are basic building blocks for both classical and quantum optical information processing. Approaching the fundamental limit of such devices, where the propagation of a single photon exhibits a good non-reciprocal characteristic, requires an asymmetric strong coupling between a single photon and a matter. Unfortunately it has been not realized yet. Here, we propose and experimentally realize a quantum non-reciprocity device with low optical losses and a high isolation of larger than 14 dB based on the cold atoms. Besides, the non-reciprocal transmission of a quantum qubit and non-reciprocal quantum storage of a true single photon are also realized. All results achieved would be very promising in building up quantum non-reciprocal devices for quantum networks.