Employing the optimal fluctuation method (OFM), we study the large deviation function of long-time averages $(1/T)int_{-T/2}^{T/2} x^n(t) dt$, $n=1,2, dots$, of centered stationary Gaussian processes. These processes are correlated and, in general, non-Markovian. We show that the anomalous scaling with time of the large-deviation function, recently observed for $n>2$ for the particular case of the Ornstein-Uhlenbeck process, holds for a whole class of stationary Gaussian processes.