We have systemically studied the effects of annealing temperature and alloy composition on the structural and magnetic properties of bulk Ni$_{2}$MnGe and Ni$_{2.1}$Mn$_{0.9}$Ge Heusler alloys. We have observed that both annealing temperature and the alloy composition drastically alter the phases found in the samples due to the presence of competing ternary phases. Annealing at 900 and 950 $^{circ}$C for both alloy compositions facilitate the formation of L2$_{1}$ Heusler phase. Nevertheless, formation of Ni$_{5}$Mn$_{4}$Ge$_{3}$ and Ni$_{16}$Mn$_{6}$Ge$_{7}$ phases cannot be prevented for Ni$_{2}$MnGe and Ni$_{2.1}$Mn$_{0.9}$Ge alloys, respectively. In order to estimate the magnetic contribution of the Ni$_{5}$Mn$_{4}$Ge$_{3}$ impurity phase to that of the parent Ni$_{2}$MnGe, we have also synthesized pure Ni$_{5}$Mn$_{4}$Ge$_{3}$ alloy. Antiferromagnetic nature of Ni$_{5}$Mn$_{4}$Ge$_{3}$ with low magnetization response allows us to reveal the magnetic response of the stoichiometric bulk Ni$_{2}$MnGe. Bulk Ni$_{2}$MnGe shows simple ferromagnetic behavior with a Curie temperature of 300 K, in agreement with the previous results on thin films. Despite the divergence of magnetization curves between field cooled (FC) and field heated (FH) modes, stoichiometric Ni$_{2}$MnGe alloy does not undergo a martensitic phase transition based on our variable temperature x-ray diffraction experiments.