We measure the spin dephasing of holes localized in self-assembled (InGa)As quantum dots by spin noise spectroscopy. The localized holes show a distinct hyperfine interaction with the nuclear spin bath despite the p-type symmetry of the valence band states. The experiments reveal a short spin relaxation time {tau}_{fast}^{hh} of 27 ns and a second, long spin relaxation time {tau}_{slow}^{hh} which exceeds the latter by more than one order of magnitude. The two times are attributed to heavy hole spins aligned perpendicular and parallel to the stochastic nuclear magnetic field. Intensity dependent measurements and numerical simulations reveal that the long relaxation time is still obscured by light absorption, despite low laser intensity and large detuning. Off-resonant light absorption causes a suppression of the spin noise signal due to the creation of a second hole entailing a vanishing hole spin polarization.