The spectrum of terahertz (THz) emission in gases via ionizing two-color femtosecond pulses is analyzed by means of a semi-analytic model and finite-difference-time-domain simulations in 1D and 2D geometries. We show that produced THz signals interac
t with free electron trajectories and thus influence significantly further THz generation upon propagation, i.e., make the process inherently nonlocal. This self-action plays a key role in the observed strong spectral broadening of the generated THz field. Diffraction limits the achievable THz bandwidth by efficiently depleting the low frequency amplitudes in the propagating field.