While defects such as oxygen vacancies in correlated materials can modify their electronic properties dramatically, understanding the microscopic origin of electronic correlations in materials with defects has been elusive. Lanthanum nickelate with oxygen vacancies, LaNiO$_{3-x}$, exhibits the metal-to-insulator transition as the oxygen vacancy level $x$ increases from the stoichiometric LaNiO$_3$. In particular, LaNiO$_{2.5}$ exhibits a paramagnetic insulating phase, also stabilizing an antiferromagnetic state below $T_Nsimeq152$K. Here, we study the electronic structure and energetics of LaNiO$_{3-x}$ using first-principles. We find that LaNiO$_{2.5}$ stabilizes a vacancy-ordered structure with an insulating ground state and the nature of the insulating phase is a site-selective paramagnetic Mott state as obtained using density functional theory plus dynamical mean field theory (DFT+DMFT). The Ni octahedron site develops a Mott insulating state with strong correlations as the Ni $e_g$ orbital is half-filled while the Ni square-planar site with apical oxygen vacancies becomes a band insulator. Our oxygen vacancy results can not be explained by the pure change of the Ni oxidation state alone within the rigid band shift approximation. Our DFT+DMFT density of states explains that the peak splitting of unoccupied states in LaNiO$_{3-x}$ measured by the experimental X-ray absorption spectra originates from two nonequivalent Ni ions in the vacancy-ordered structure.