There are a number of very high energy sources in the Galaxy that remain unidentified. Multi-wavelength and variability studies, and catalogue searches, are powerful tools to identify the physical counterpart, given the uncertainty in the source location and extension. This work carries out a thorough multi-wavelength study of the unidentified, very high energy source HESS J1858+020 and its environs. Giant Metrewave Radio Telescope observations at 610 MHz and 1.4 GHz have been done to obtain a deep, low-frequency radio image of the region surrounding HESS J1858+020. Archival radio, infrared, and X-ray data have been analysed as well. This observational information, combined with molecular data, catalogue sources, and a nearby Fermi gamma-ray detection of unidentified origin, are combined to explore possible counterparts to the very high energy source. We provide with a deep radio image of a supernova remnant that might be related to the GeV and TeV emission in the region. We confirm the presence of an H II region next to the supernova remnant and coincident with molecular emission. A potential region of star formation is also identified. We identify several radio and X-ray sources in the surroundings. Some of these sources are known planetary nebulae, whereas others may be non-thermal extended emitters and embedded young stellar objects. Three old, background Galactic pulsars also neighbour HESS J1858+020 along the line of sight. The region surrounding HESS J1858+020 is rich in molecular structures and non-thermal objects that may potentially be linked to this unidentified very high energy source. In particular, a supernova remnant interacting with nearby molecular clouds may be a good candidate, but a star forming region, or a non-thermal radio source of yet unclear nature, may also be behind the gamma-ray source. Further observational studies are needed.