In this paper, we generally expressed the virial expansion of ideal quantum gases by the heat kernel coefficients for the corresponding Laplace type operator. As examples, we give the virial coefficients for quantum gases in $d$-dimensional confined space and spheres, respectively. Our results show that, the relative correction from the boundary to the second virial coefficient is independent of the dimension and it always enhances the quantum exchange interaction. In $d$-dimensional spheres, however, the influence of the curvature enhances the quantum exchange interaction in two dimensions, but weakens it in higher dimensions ($d>3$).