We analyze the wavelength-dependent variability of a sample of spectroscopically confirmed active galactic nuclei (AGN) selected from near-UV ($NUV$) variable sources in the GALEX Time Domain Survey that have a large amplitude of optical variability (difference-flux S/N $>$ 3) in the Pan-STARRS1 Medium Deep Survey (PS1 MDS). By matching GALEX and PS1 epochs in 5 bands ($NUV$, $g_{P1}$, $r_{P1}$, $i_{P1}$, $z_{P1}$) in time, and taking their flux difference, we create co-temporal difference-flux spectral energy distributions ($Delta f$SEDs) using two chosen epochs for each of the 23 objects in our sample on timescales of about a year. We confirm the bluer-when-brighter trend reported in previous studies, and measure a median spectral index of the $Delta f$SEDs of $alpha_{lambda}$ = 2.1 that is consistent with an accretion disk spectrum. We further fit the $Delta f$SEDs of each source with a standard accretion disk model in which the accretion rate changes from one epoch to the other. In our sample, 17 out of 23 ($sim$74 %) sources are well described by this variable accretion-rate disk model, with a median average characteristic disk temperature $bar{T}^*$ of $1.2times 10^5$~K that is consistent with the temperatures expected given the distribution of accretion rates and black hole masses inferred for the sample. Our analysis also shows that the variable accretion rate model is a better fit to the $Delta f$SEDs than a simple power law.