The Zagier $L$-series encode data of real quadratic fields. We study the average size of these $L$-series, and prove asymptotic expansions and omega results for the expansion. We then show how the error term in the asymptotic expansion can be used to obtain error terms in the prime geodesic theorem.