Ion beam irradiation has recently emerged as a versatile approach to functional materials design. We show in this work that patterned defective regions generated by ion beam irradiation of silicon can create a phonon glass electron crystal (PGEC), a longstanding goal of thermoelectrics. By controlling the effective diameter of and spacing between the defective regions, molecular dynamics simulations suggest a reduction of the thermal conductivity by a factor of $approx$20 is achievable. Boltzmann theory shows that the thermoelectric power factor remains largely intact in the damaged material. To facilitate the Boltzmann theory, we derive an analytical model for electron scattering with cylindrical defective regions based on partial wave analysis. Together we predict a figure of merit of ZT$approx$0.5 or more at room temperature for optimally patterned geometries of these silicon metamaterials. These findings indicate that nanostructuring of patterned defective regions in crystalline materials is a viable approach to realize a PGEC, and ion beam irradiation could be a promising fabrication strategy.