By a series of simple examples, we illustrate how the lack of mathematical concern can readily lead to surprising mathematical contradictions in wave mechanics. The basic mathematical notions allowing for a precise formulation of the theory are then summarized and it is shown how they lead to an elucidation and deeper understanding of the aforementioned problems. After stressing the equivalence between wave mechanics and the other formulations of quantum mechanics, i.e. matrix mechanics and Diracs abstract Hilbert space formulation, we devote the second part of our paper to the latter approach: we discuss the problems and shortcomings of this formalism as well as those of the bra and ket notation introduced by Dirac in this context. In conclusion, we indicate how all of these problems can be solved or at least avoided.