In this paper, we study a series of $L^2$-torsion invariants from the viewpoint of the mapping class group of a surface. We establish some vanishing theorems for them. Moreover we explicitly calculate the first two invariants and compare them with hyperbolic volumes.