We present a study of the spectral linewidth of collimated blue light (CBL) that results from wave mixing of low-power cw laser radiation at 780 nm and 776 nm and an internally-generated mid-IR field at 5.23 um in Rb vapour. Using a high-finesse Fabry-Perot interferometer the spectral width of the CBL is found to be less than 1.3 MHz for a wide range of experimental conditions. We demonstrate that the CBL linewidth is mainly limited by the temporal coherence of the applied laser fields rather than the atom-light interaction itself. Results obtained with frequency modulated laser light allow an upper limit of several hundred kHz to be set for the linewidth of the collimated mid-IR radiation at 5.23 um, which has not been directly detected.