GALAH+ is a magnitude-limited survey of high resolution stellar spectra obtained by the HERMES spectrograph at the Australian Astronomical Observatory. Its third data release provides reduced spectra with new derivations of stellar parameters and abundances of 30 chemical elements for 584,015 dwarfs and giants, 88% of them in the Gaia magnitude range 11 < G < 14. Here we use these improved values of stellar parameters to build a library of observed spectra which is useful to study variations of individual spectral lines with stellar parameters. This and other improvements are used to derive radial velocities with uncertainties which are generally within 0.1 km/s or ~25% smaller than in the previous release. Median differences in radial velocities measured here and by the Gaia DR2 or APOGEE DR16 surveys are smaller than 30 m/s, a larger offset is present only for Gaia measurements of giant stars. We identify 4483 stars with intrinsically variable velocities and 225 stars for which the velocity stays constant over >=3 visits spanning more than a year. The combination of radial velocities from GALAH+ with distances and sky plane motions from Gaia enables studies of dynamics within streams and clusters. For example, we estimate that the open cluster M67 has a total mass of ~3300 Msun and its outer parts seem to be expanding, though astrometry with a larger time-span than currently available from Gaia eDR3 is needed to judge if the latter result is real.