We introduce a scattering-type scanning near-field infrared microscope (s-SNIM) for the local scale near- field sample analysis and spectroscopy from room (RT) down to liquid helium (LHe) temperatures. The extension of s-SNIM down to T = 5K is in particular crucial for low-temperature phase transitions, e.g. for the examination of superconductors, as well as low energy excitations. The LT s-SNIM performance is tested with CO2-IR excitation at T = 7K using a bare Au reference and a structured Si/SiO$_2$-sample. Furthermore, we quantify the impact of local laser heating under the s-SNIM tip apex by monitoring the light-induced ferroelectric-to-paraelectric phase transition of the skyrmion-hosting multiferroic material GaV4S8 at T$_c$ = 42K. We apply LT s-SNIM to study the spectral response of GaV$_4$S$_8$ and its lateral domain structure in the ferroelectric phase by the mid-IR to THz free-electron laser-light source FELBE at the Helmholtz-Zentrum Dresden-Rossendorf, Germany. Notably, our s-SNIM is based on a non-contact atomic force microscope(AFM), and thus can be complemented in-situ by various other AFM techniques, such as topography profiling, piezo-response force microscopy (PFM) and/or Kelvin-probe force microscopy (KPFM). The combination of these methods support the comprehensive study of the mutual interplay in the topographic, electronic and optical properties of surfaces from room temperature down to 5K.