We have demonstrated experimentally the manipulation of exciton and nuclear spins in a single self-assembled In$_{0.75}$Al$_{0.25}$As/Al$_{0.3}$Ga$_{0.7}$As quantum dot. The oscillation of exciton and nuclear spin polarizations were clearly observed. The switching of the emissions in Zeeman split pair indicates that the exciton pair with opposite spins was created coherently via the continuum states and that we can control the electron and nuclear spin polarizations only by changing the delay time of the cross-linearly-polarized pulses. These suggest the high potentiality of electron and nuclear spin manipulation in a single QD via the continuum state.