We give the exact expressions of the partial susceptibilities $chi^{(3)}_d$ and $chi^{(4)}_d$ for the diagonal susceptibility of the Ising model in terms of modular forms and Calabi-Yau ODEs, and more specifically, $_3F_2([1/3,2/3,3/2],, [1,1];, z)$ and $_4F_3([1/2,1/2,1/2,1/2],, [1,1,1]; , z)$ hypergeometric functions. By solving the connection problems we analytically compute the behavior at all finite singular points for $chi^{(3)}_d$ and $chi^{(4)}_d$. We also give new results for $chi^{(5)}_d$. We see in particular, the emergence of a remarkable order-six operator, which is such that its symmetric square has a rational solution. These new exact results indicate that the linear differential operators occurring in the $n$-fold integrals of the Ising model are not only Derived from Geometry (globally nilpotent), but actually correspond to Special Geometry (homomorphic to their formal adjoint). This raises the question of seeing if these special geometry Ising-operators, are special ones, reducing, in fact systematically, to (selected, k-balanced, ...) $_{q+1}F_q$ hypergeometric functions, or correspond to the more general solutions of Calabi-Yau equations.