Recently synthesized metastable tetragonal CoSe, isostructural to the FeSe superconductor, offers a new avenue for investigating systems in close proximity to the iron-based superconductors. We present magnetic and transport property measurements on powders and single crystals of CoSe. High field magnetic susceptibility measurements indicate a suppression of the previously reported 10 K ferromagnetic transition with the magnetic susceptibility exhibiting time-dependence below the proposed transition. Dynamic scaling analysis of the time-dependence yields a critical relaxation time of $tau^{*} = 0.064 pm 0.008 $ s which in turn yields an activation energy of $E_{a}^{*}$ = 14.84 $pm$ 0.59 K and an ideal glass temperature $T_{0}^{*}$ = 8.91 $pm$ 0.09 K from Vogel-Fulcher analysis. No transition is observed in resistivity and specific heat measurements, but both measurements indicate that CoSe is metallic. These results are interpreted on the basis of CoSe exhibiting frustrated magnetic ordering arising from competing magnetic interactions. Arrott analysis of single crystal magnetic susceptibility has indicated that the magnetic moments lie in the $ab$-plane so frustration may arise from intralayer magnetic fluctuations as well as interlayer coupling. The results have implications for understanding the superconductivity in the iron chalcogenide systems as well as utilizing CoSe as a host for chemical and physical manipulation to tune and explore emergent phenomena within an expanding new class of transition metal chalcogenides.