We have derived estimators for the linear growth rate of density fluctuations using the cross-correlation function of voids and haloes in redshift space, both directly and in Fourier form. In linear theory, this cross-correlation contains only monopole and quadrupole terms. At scales greater than the void radius, linear theory is a good match to voids traced out by haloes in N-body simulations; small-scale random velocities are unimportant at these radii, only tending to cause small and often negligible elongation of the redshift-space cross-correlation function near its origin. By extracting the monopole and quadrupole from the cross-correlation function, we measure the linear growth rate without prior knowledge of the void profile or velocity dispersion. We recover the linear growth parameter $beta$ to 9% precision from an effective volume of 3(Gpc/h)^3 using voids with radius greater than 25Mpc/h. Smaller voids are predominantly sub-voids, which may be more sensitive to the random velocity dispersion; they introduce noise and do not help to improve the measurement. Adding velocity dispersion as a free parameter allows us to use information at radii as small as half of the void radius. The precision on $beta$ is reduced to approximately 5%. Contrary to the simple redshift-space distortion pattern in overdensities, voids show diverse shapes in redshift space, and can appear either elongated or flattened along the line of sight. This can be explained by the competing amplitudes of the local density contrast, plus the radial velocity profile and its gradient, with the latter two factors being determined by the cumulative density profile of voids. The distortion pattern is therefore determined solely by the void profile and is different for void-in-cloud and void-in-void. This diversity of redshift-space void morphology complicates measurements of the Alcock-Paczynski effect using voids.