Large thermal hysteresis in the MnFe(P, Si, B) system hinders the heat exchange rate and thus limits the magnetocaloric applications at high frequencies. Substitution of Mn by V in Mn1-xVxFe0.95P0.593Si0.33B0.077 and Mn1-xVxFe0.95P0.563Si0.36B0.077 alloys was found to reduce the thermal hysteresis due to a decrease in the latent heat. Introducing V increases both the field-induced transition temperature shift and the magnetic moment per formula unit. Thus, a decease in the thermal hysteresis is obtained without losing the giant magnetocaloric effect. In consequence, an ultralow hysteresis (0.7 K) and a giant adiabatic temperature change of 2.3 K were achieved, which makes these alloys promising candidates for commercial magnetic refrigerator using permanent magnets.