We prove new upper bounds for a spectral exponential sum by refining the process by which one evaluates mean values of $L$-functions multiplied by an oscillating function. In particular, we introduce a method which is capable of taking into consideration the oscillatory behaviour of the function. This gives an improvement of the result of Luo and Sarnak when $Tgeq X^{1/6+2theta/3}$. Furthermore, this proves the conjecture of Petridis and Risager in some ranges. Finally, this allows obtaining a new proof of the Soundararajan-Young error estimate in the prime geodesic theorem.