We study the impact of the spectral dependence of the linear polarization rotation induced by an achromatic half-wave plate on measurements of cosmic microwave background polarization in the presence of astrophysical foregrounds. We focus on the systematic effects induced on the measurement of inflationary gravitational waves by uncertainties in the polarization and spectral index of Galactic dust. We find that for the experimental configuration and noise levels of the balloon-borne EBEX experiment, which has three frequency bands centered at 150, 250, and 410 GHz, a crude dust subtraction process mitigates systematic effects to below detectable levels for 10% polarized dust and tensor to scalar ratio of as low as r = 0.01. We also study the impact of uncertainties in the spectral response of the instrument. With a top-hat model of the spectral response for each band, characterized by band-center and band-width, and with the same crude dust subtraction process, we find that these parameters need to be determined to within 1 and 0.8 GHz at 150 GHz; 9 and 2.0 GHz at 250 GHz; and 20 and 14 GHz at 410 GHz, respectively. The approach presented in this paper is applicable to other optical elements that exhibit polarization rotation as a function of frequency.