We use first-principles calculations to investigate the stability of bi-axially strained textit{Pnma} perovskite CaMnO$_3$ towards the formation of oxygen vacancies. Our motivation is provided by promising indications that novel material properties can be engineered by application of strain through coherent heteroepitaxy in thin films. While it is usually assumed that such epitaxial strain is accommodated primarily by changes in intrinsic lattice constants, point defect formation is also a likely strain relaxation mechanism. This is particularly true at the large strain magnitudes ($>$4%) which first-principles calculations often suggest are required to induce new functionalities. We find a strong dependence of oxygen vacancy defect formation energy on strain, with tensile strain lowering the formation energy consistent with the increasing molar volume with increasing oxygen deficiency. In addition, we find that strain differentiates the formation energy for different lattice sites, suggesting its use as a route to engineering vacancy ordering in epitaxial thin films.