We report the hysteresis of optically-pumped nuclear spin polarization and the degree of circular polarization of photoluminescence on the excitation power and electron spin polarization in single InAlAs quantum dots. By increasing (or decreasing) the excitation power at a particular excitation polarization, an abrupt rise (or drop) and a clear hysteretic behavior were observed in the Overhauser shift of the photoluminescence of the exciton and exciton complexes from the same single quantum dot under an external magnetic field of 5 T. However, the degree of circular polarization shows different behaviors between a positively charged exciton and a neutral exciton or biexciton; further, only positively charged exciton exhibits the precisely synchronized change and hysteretic behavior. It is suggested that the electron spin distribution is affected by the flip-flop of electron-nuclear spins. Further, the hysteresis is observed as a function of the degree of circular polarization of the excitation light and its dependence on the excitation power is studied. The saturation of the Overhauser shift after the abrupt rise indicates the almost complete cancellation of the external magnetic field by the nuclear field created within the width that is decided by the correlation time between the electron and the nuclei spin system.