We developed a calorimeter with a vacuum container made of superconducting niobium (Nb) to study monolayers of helium adsorbed on graphite which are prototypical two-dimensional quantum matters below 1 K. Nb was chosen because of its small specific heat in the superconducting state. It is crucially important to reduce the addendum heat capacity ($C_{rm{ad}}$) when the specific surface area of substrate is small. Here we show details of design, construction and results of $C_{rm{ad}}$ measurements of the Nb calorimeter down to 40 mK. The measured $C_{rm{ad}}$ was sufficiently small so that we can use it for heat capacity measurements on helium monolayers in a wide temperature range below 1 K. We found a relatively large excess heat capacity in $C_{rm{ad}}$, which was successfully attributed to atomic tunneling of hydrogen (H) and deuterium (D) between trap centers near oxygen or nitrogen impurities in Nb. The tunnel frequencies of H and D deduced by fitting the data to the tunneling model are consistent with the previous experiments on Nb doped with H or D.