The $gamma$ vibration is the most typical low-lying collective motion prevailing the nuclear chart. But only few one-phonon rotational bands in odd-$A$ nuclei have been known. Furthermore, two-phonon states, even the band head, have been observed in a very limited number of nuclides not only of odd-$A$ but even-even. Among them, that in $^{105}$Mo is unique in that Coriolis effects are expected to be stronger than in $^{103}$Nb and $^{105}$Nb on which theoretical studies were reported. Then the purpose of the present work is to study $^{105}$Mo paying attention to rotational character change of the one-phonon and two-phonon bands. The particle-vibration coupling model based on the cranking model and the random-phase approximation is used to calculate the vibrational states in rotating odd-$A$ nuclei. The present model reproduces the observed yrast zero-phonon and one-phonon bands well. Emerging general features of the rotational character change from low spin to high spin are elucidated. In particular, the reason why the one-phonon band does not exhibit signature splitting is clarified. The calculated collectivity of the two-phonon states, however, is located higher than observed.