We have investigated spin and carrier dynamics of resident holes in high-mobility two-dimensional hole systems in GaAs/Al$_{0.3}$Ga$_{0.7}$As single quantum wells at temperatures down to 400 mK. Time-resolved Faraday and Kerr rotation, as well as time-resolved photoluminescence spectroscopy are utilized in our study. We observe long-lived hole spin dynamics that are strongly temperature dependent, indicating that in-plane localization is crucial for hole spin coherence. By applying a gate voltage, we are able to tune the observed hole g factor by more than 50 percent. Calculations of the hole g tensor as a function of the applied bias show excellent agreement with our experimental findings.