(abridged) We calculate near-infrared thermal emission spectra using a doubling-adding radiative transfer code, which includes scattering by clouds and haze. Initial temperature profiles and cloud optical depths are taken from the drift-phoenix brown dwarf model. As is well known, cloud particles change the spectrum compared to when clouds are ignored. The clouds reduce fluxes in the near-infrared spectrum and make it redder than for the clear sky case. We also confirm that not including scattering in the spectral calculations can result in errors on the spectra of many tens of percent, both in magnitude and in variations with wavelength. This is especially apparent for particles that are larger than the wavelength and only have little iron in them. Scattering particles will show deeper absorption features than absorbing (e.g. iron) particles and particle size will also affect the calculated infrared colours. Large particles also tend to be strongly forward-scattering, and we show that assuming isotropic scattering in this case also leads to very large errors in the spectrum. Thus, care must be taken in the choice of radiative transfer method for heat balance or spectral calculations when clouds are present in the atmosphere. Besides the choice of radiative transfer method, the type of particles that are predicted by models will change conclusions about e.g. infrared colours and trace gas abundances. As a result, knowledge of the scattering properties of the clouds is essential when deriving temperature profiles or gas abundances from direct infrared observations of exoplanets or brown dwarfs and from secondary eclipse measurements of transiting exoplanets, since scattering clouds will change the depth of gas absorption features, among other things. Thus, ignoring the presence of clouds can yield retrieved properties that differ significantly from the real atmospheric properties.