The low sensitivity of photons to external magnetic fields is one of the major challenges for the engineering of photonic lattices with broken time-reversal symmetry. Here we show that time-reversal symmetry can be broken for microcavity polaritons in the absence of any external magnetic field thanks to polarization dependent polariton interactions. Circularly polarized excitation of carriers in a micropillar induces a Zeeman-like energy splitting between polaritons of opposite polarizations. In combination with optical spin-orbit coupling inherent to semiconductor microstructures, the interaction induced Zeeman splitting results in emission of vortical beams with a well-defined chirality. Our experimental findings can be extended to lattices of coupled micropillars opening the possibility of controling optically the topological properties of polariton Chern insulators.