We perform, as a function of uniaxial stress, an optical-reflectivity investigation of the representative parent ferropnictide BaFe$_2$As$_2$ in a broad spectral range, across the tetragonal-to-orthorhombic phase transition and the onset of the long-range antiferromagnetic order (AFM). The infrared response reveals that the $dc$ transport anisotropy in the orthorhombic AFM state is determined by the interplay between the Drude spectral weight and the scattering rate, but that the dominant effect is clearly associated with the metallic spectral weight. In the paramagnetic tetragonal phase, though, the $dc$ resistivity anisotropy of strained samples is almost exclusively due to stress-induced changes in the Drude weight rather than in the scattering rate, definitively establishing the anisotropy of the Fermi surface parameters as the primary effect driving the $dc$ transport properties in the electronic nematic state.