The idea of modular invariance provides a novel explanation of flavour mixing. Within the context of finite modular symmetries $Gamma_N$ and for a given element $gamma in Gamma_N$, we present an algorithm for finding stabilisers (specific values for moduli fields $tau_gamma$ which remain unchanged under the action associated to $gamma$). We then employ this algorithm to find all stabilisers for each element of finite modular groups for $N=2$ to $5$, namely, $Gamma_2simeq S_3$, $Gamma_3simeq A_4$, $Gamma_4simeq S_4$ and $Gamma_5simeq A_5$. These stabilisers then leave preserved a specific cyclic subgroup of $Gamma_N$. This is of interest to build models of fermionic mixing where each fermionic sector preserves a separate residual symmetry.