We study properties of the full partition function for the $U(1)$ 5D $mathcal{N}=2^*$ gauge theory with adjoint hypermultiplet of mass $M$. This theory is ultimately related to abelian 6D (2,0) theory. We construct the full non-perturbative partition function on toric Sasaki-Einstein manifolds by gluing flat copies of the Nekrasov partition function and we express the full partition function in terms of the generalized double elliptic gamma function $G_2^C$ associated with a certain moment map cone $C$. The answer exhibits a curious $SL(4,mathbb{Z})$ modular property. Finally, we propose a set of rules to construct the partition function that resembles the calculation of 5D supersymmetric partition function with the insertion of defects of various co-dimensions.