Computing the ground-state properties of quantum many-body systems is a promising application of near-term quantum hardware with a potential impact in many fields. Quantum phase estimation uses deep circuits and is infeasible without fault-tolerant technologies. Many quantum simulation algorithms developed recently work in an inexact and variational manner to exploit the power of shallow circuits. These algorithms rely on the assumption that variational circuits can produce the desired result. Here, we combine quantum Monte Carlo with quantum computing and propose a quasi-exact algorithm for imaginary-time simulation and ground-state computing. Unlike variational algorithms, our algorithm always approaches the exact solution when the Trotter circuit depth increases. Even when the circuit is shallow, our algorithm can yield an accurate ground-state energy. Compared with quantum phase estimation, the conventional quasi-exact algorithm, our algorithm can reduce the Trotter step number by thousands of times. We verify this resilience to Trotterisation errors in numerical simulation of up to 20 qubits and theoretical analysis. Our results demonstrate that non-variational and exact quantum simulation is promising even without a fully fault-tolerant quantum computer.