Umbral moonshine connects the symmetry groups of the 23 Niemeier lattices with 23 sets of distinguished mock modular forms. The 23 cases of umbral moonshine have a uniform relation to symmetries of $K3$ string theories. Moreover, a supersymmetric vertex operator algebra with Conway sporadic symmetry also enjoys a close relation to the $K3$ elliptic genus. Inspired by the above two relations between moonshine and $K3$ string theory, we construct a chiral CFT by orbifolding the free theory of 24 chiral fermions and two pairs of fermionic and bosonic ghosts. In this paper we mainly focus on the case of umbral moonshine corresponding to the Niemeier lattice with root system given by 6 copies of $D_4$ root system. This CFT then leads to the construction of an infinite-dimensional graded module for the umbral group $G^{D_4^{oplus 6}}$ whose graded characters coincide with the umbral moonshine functions. We also comment on how one can recover all umbral moonshine functions corresponding to the Niemeier root systems $A_5^{oplus 4}D_4$, $A_7^{oplus 2}D_5^{oplus 2}$ , $A_{11}D_7 E_6$, $A_{17}E_7$, and $D_{10}E_7^{oplus 2}$.