Ferrimagnets, which contain the advantages of both ferromagnets (detectable moments) and antiferromagnets (ultrafast spin dynamics), have recently attracted great attention. Here we report the optimization of epitaxial growth of a tetragonal perpendicularly magnetized ferrimagnet Mn2Ga on MgO. Electrical transport, magnetic properties and the anomalous Hall effect (AHE) were systematically studied. Furthermore, we successfully integrated high-quality epitaxial ferrimagnetic Mn2Ga thin films onto ferroelectric PMN-PT single crystals with a MgO buffer layer. It was found that the AHE of such a ferrimagnet can be effectively modulated by a small electric field over a large temperature range in a nonvolatile manner. This work thus demonstrates the great potential of ferrimagnets for developing high-density and low-power spintronic devices.