We synthesized a series of V-doped LiFe$_{1-x}$V$_x$As single crystals. The superconducting transition temperature $T_c$ of LiFeAs decreases rapidly at a rate of 7 K per 1% V. The Hall coefficient of LiFeAs switches from negative to positive with 4.2% V doping, showing that V doping introduces hole carriers. This observation is further confirmed by the evaluation of the Fermi surface volume measured by angle-resolved photoemission spectroscopy (ARPES), from which a 0.3 hole doping per V atom introduced is deduced. Interestingly, the introduction of holes does not follow a rigid band shift. We also show that the temperature evolution of the electrical resistivity as a function of doping is consistent with a crossover from a Fermi liquid to a non-Fermi liquid. Our ARPES data indicate that the non-Fermi liquid behavior is mostly enhanced when one of the hole $d_{xz}/d_{yz}$ Fermi surfaces is well nested by the antiferromagnetic wave vector to the inner electron Fermi surface pocket with the $d_{xy}$ orbital character. The magnetic susceptibility of LiFe$_{1-x}$V$_x$As suggests the presence of strong magnetic impurities following V doping, thus providing a natural explanation to the rapid suppression of superconductivity upon V doping.