The H1 photonic crystal cavity supports two degenerate dipole modes of orthogonal linear polarization which could give rise to circularly polarized fields when driven with a $pi$/$2$ phase difference. However, fabrication errors tend to break the symmetry of the cavity which lifts the degeneracy of the modes, rendering the cavity unsuitable for supporting circular polarization. We demonstrate numerically, a scheme that induces chirality in the cavity modes, thereby achieving a cavity that supports intrinsic circular polarization. By selectively modifying two air holes around the cavity, the dipole modes could interact via asymmetric coherent backscattering which is a non-Hermitian process. With suitable air hole parameters, the cavity modes approach the exceptional point, coalescing in frequencies and linewidths as well as giving rise to significant circular polarization close to unity. The handedness of the chirality can be selected depending on the choice of the modified air holes. Our results highlight the prospect of using the H1 photonic crystal cavity for chiral-light matter coupling in applications such as valleytronics, spin-photon interfaces and the generation of single photons with well-defined spins.