We consider a description of the stochastic oscillations of the general relativistic accretion disks around compact astrophysical objects interacting with their external medium based on a generalized Langevin equation with colored noise, which accounts for the general memory and retarded effects of the frictional force, and on the fluctuation-dissipation theorem. The presence of the memory effects influences the response of the disk to external random interactions, and modifies the dynamical behavior of the disk, as well as the energy dissipation processes. The generalized Langevin equation of the motion of the disk in the vertical direction is studied numerically, and the vertical displacements, velocities and luminosities of the stochastically perturbed disks are explicitly obtained for both the Schwarzschild and the Kerr cases. The Power Spectral Distribution (PSD) of the disk luminosity is also obtained. As a possible astrophysical application of the formalism we investigate the possibility that the Intra Day Variability (IDV) of the Active Galactic Nuclei (AGN) may be due to the stochastic disk instabilities. The perturbations due to colored/nontrivially correlated noise induce a complicated disk dynamics, which could explain some astrophysical observational features related to disk variability.