Majorana bound states appearing in 1-D $p$-wave superconductor ($cal{PWS}$) are found to result in exotic quantum holonomy of both eigenvalues and the eigenstates. Induced by a degeneracy hidden in complex Bloch vector space, Majorana states are identified with a pair of exceptional point ($cal{EP}$) singularities. Characterized by a collapse of the vector space, these singularities are defects in Hilbert space that lead to M$ddot{rm o}$bius strip-like structure of the eigenspace and singular quantum metric. The topological phase transition in the language of $cal{EP}$ is marked by one of the two exception point singularity degenerating to a degeneracy point with non singular quantum metric. This may provide an elegant and useful framework to characterize the topological aspect of Majorana fermions and the topological phase transition.