We study the dependence of mild solutions to linear stochastic evolution equations on Hilbert space driven by Wiener noise, with drift having linear part of the type $A+varepsilon G$, on the parameter $varepsilon$. In particular, we study the limit and the asymptotic expansions in powers of $varepsilon$ of these solutions, as well as of functionals thereof, as $varepsilon to 0$, with good control on the remainder. These convergence and series expansion results are then applied to a parabolic perturbation of the Musiela SPDE of mathematical finance modeling the dynamics of forward rates.