Motivated by recent development in quantum fidelity and fidelity susceptibility, we study relations among Lie algebra, fidelity susceptibility and quantum phase transition for a two-state system and the Lipkin-Meshkov-Glick model. We get the fidelity susceptibility for SU(2) and SU(1,1) algebraic structure models. From this relation, the validity of the fidelity susceptibility to signal for the quantum phase transition is also verified in these two systems. At the same time, we obtain the geometric phase in these two systems in the process of calculating the fidelity susceptibility. In addition, the new method of calculating fidelity susceptibility has been applied to explore the two-dimensional XXZ model and the Bose-Einstein condensate(BEC).