Extending recent work on stress fluctuations in complex fluids and amorphous solids we describe in general terms the ensemble average $v(Delta t)$ and the standard deviation $delta v(Delta t)$ of the variance $v[mathbf{x}]$ of time series $mathbf{x}$ of a stochastic process $x(t)$ measured over a finite sampling time $Delta t$. Assuming a stationary, Gaussian and ergodic process, $delta v$ is given by a functional $delta v_G[h]$ of the autocorrelation function $h(t)$. $delta v(Delta t)$ is shown to become large and similar to $v(Delta t)$ if $Delta t$ corresponds to a fast relaxation process. Albeit $delta v = delta v_G[h]$ does not hold in general for non-ergodic systems, the deviations for common systems with many microstates are merely finite-size corrections. Various issues are illustrated for shear-stress fluctuations in simple coarse-grained model systems.