Optical nonreciprocity is important in photonic information processing to route the optical signal or prevent the reverse flow of noise. By adopting the strong nonlinearity associated with a few atoms in a strongly coupled cavity QED system and an asymmetric cavity configuration, we experimentally demonstrate the nonreciprocal transmission between two counterpropagating light fields with extremely low power. This nonreciprocity can even occur on a few-photon level due to the high optical nonlinearity of the system. The working power can be flexibly tuned by changing the effective number of atoms strongly coupled to the cavity. The idea and result can be applied to optical chips as optical diodes by using fiber-based cavity QED systems. Our work opens up new perspectives for realizing optical nonreciprocity on a few-photon level based on the nonlinearities of atoms strongly coupled to an optical cavity.