The symmetry operators generating the hidden $mathbb{Z}_2$ symmetry of the asymmetric quantum Rabi model (AQRM) at bias $epsilon in frac{1}{2}mathbb{Z}$ have recently been constructed by V. V. Mangazeev et al. [J. Phys. A: Math. Theor. 54 12LT01 (2021)]. We start with this result to determine symmetry operators for the $N$-qubit generalisation of the AQRM, also known as the biased Dicke model, at special biases. We also prove for general $N$ that the symmetry operators, which commute with the Hamiltonian of the biased Dicke model, generate a $mathbb{Z}_2$ symmetry.