The relevant interaction energies for astrophysical radiative capture reactions are very low, much below the repulsive Coulomb barrier. This leads to low cross sections, low counting rates in $gamma$-ray detectors, and therefore the need to perform such experiments at ion accelerators placed in underground settings, shielded from cosmic rays. Here, the feasibility of such experiments in the new shallow-underground accelerator laboratory in tunnels VIII and IX of the Felsenkeller site in Dresden, Germany, is evaluated. To this end, the no-beam background in three different types of germanium detectors, i.e. a Euroball/Miniball triple cluster and two large monolithic detectors, is measured over periods of 26-66 days. The cosmic-ray induced background is found to be reduced by a factor of 500-2400, by the combined effects of, first, the 140 meters water equivalent overburden attenuating the cosmic muon flux by a factor of 40, and second, scintillation veto detectors gating out most of the remaining muon-induced effects. The new background data are compared to spectra taken with the same detectors at the Earths surface and at other underground sites. Subsequently, the beam intensity from the cesium sputter ion source installed in Felsenkeller has been studied over periods of several hours. Based on the background and beam intensity data reported here, for the example of the $^{12}$C($alpha$,$gamma$)$^{16}$O reaction it is shown that highly sensitive experiments will be possible.