A main distinguishing feature of non-Hermitian quantum mechanics is the presence of exceptional points (EPs). They correspond to the coalescence of two energy levels and their respective eigenvectors. Here, we use the Lipkin-Meshkov-Glick (LMG) model as a testbed to explore the strong connection between EPs and the onset of excited state quantum phase transitions (ESQPTs). We show that for finite systems, the exact degeneracies (EPs) obtained with the non-Hermitian LMG Hamiltonian continued into the complex plane are directly linked with the avoided crossings that characterize the ESQPTs for the real (physical) LMG Hamiltonian. The values of the complex control parameter $alpha$ that lead to the EPs approach the real axis as the system size $Nrightarrow infty$. This happens for both, the EPs that are close to the separatrix that marks the ESQPT and also for those that are far away, although in the latter case, the rate the imaginary part of $alpha$ reduces to zero as $N$ increases is smaller. With the method of Pade approximants, we can extract the critical value of $alpha$.