The unique electronic structure found at interfaces between materials can allow unconventional quantum states to emerge. Here we observe superconductivity in electron gases formed at interfaces between (111) oriented KTaO$_3$ and insulating overlayers of either EuO or LaAlO$_3$. The superconducting transition temperature, approaching 2.2 K, is about one order of magnitude higher than that of the LaAlO$_3$/SrTiO$_3$ system. Strikingly, similar electron gases at (001) KTaO$_3$ interfaces remain normal down to 25 mK. The critical field and current-voltage measurements indicate that the superconductivity is two dimensional. Higher mobility EuO/KTaO$_3$ (111) samples show a large in-plane anisotropy in transport properties at low temperatures prior to onset of superconductivity, suggesting the emergence of a stripe like phase where the superconductivity is nearly homogeneous in one direction, but strongly modulated in the other.