Due to its p-like character, the valence band in GaAs-based heterostructures offers rich and complex spin-dependent phenomena. One manifestation is the large anisotropy of Zeeman spin splitting. Using undoped, coupled quantum wells (QWs), we examine this anisotropy by comparing the hole spin dynamics for high- and low-symmetry crystallographic orientations of the QWs. We directly measure the hole $g$ factor via time-resolved Kerr rotation, and for the low-symmetry crystallographic orientations (110) and (113a), we observe a large in-plane anisotropy of the hole $g$ factor, in good agreement with our theoretical calculations. Using resonant spin amplification, we also observe an anisotropy of the hole spin dephasing in the (110)-grown structure, indicating that crystal symmetry may be used to control hole spin dynamics.