We propose quenched disorders could bring novel quantum excitations and models to certain quantum magnets. Motivated by the recent experiments on the quantum Ising magnet TmMgGaO$_4$, we explore the effects of the quenched disorder and the interlayer coupling in this triangular lattice Ising antiferromagnet. It is pointed out that the weak quenched (non-magnetic) disorder would convert the emergent 2D Berezinskii-Kosterlitz-Thouless (BKT) phase and the critical region into a gauge glass. There will be an emergent Halperin-Saslow mode associated with this gauge glass. Using the Imry-Ma argument, we further explain the fate of the finite-field $C_3$ symmetry breaking transition at the low temperatures. The ferromagnetic interlayer coupling would suppress the BKT phase and generate a tiny ferromagnetism. With the quenched disorders, this interlayer coupling changes the 2D gauge glass into a 3D gauge glass, and the Halperin-Saslow mode persists. This work merely focuses on addressing a phase regime in terms of emergent U(1) gauge glass behaviors and hope to inspire future works and thoughts in weakly disordered frustrated magnets in general.