Halfspaces or linear threshold functions are widely studied in complexity theory, learning theory and algorithm design. In this work we study the natural problem of constructing pseudorandom generators (PRGs) for halfspaces over the sphere, aka spherical caps, which besides being interesting and basic geometric objects, also arise frequently in the analysis of various randomized algorithms (e.g., randomized rounding). We give an explicit PRG which fools spherical caps within error $epsilon$ and has an almost optimal seed-length of $O(log n + log(1/epsilon) cdot loglog(1/epsilon))$. For an inverse-polynomially growing error $epsilon$, our generator has a seed-length optimal up to a factor of $O( log log {(n)})$. The most efficient PRG previously known (due to Kane, 2012) requires a seed-length of $Omega(log^{3/2}{(n)})$ in this setting. We also obtain similar constructions to fool halfspaces with respect to the Gaussian distribution. Our construction and analysis are significantly different from previous works on PRGs for halfspaces and build on the iterative dimension reduction ideas of Kane et. al. (2011) and Celis et. al. (2013), the emph{classical moment problem} from probability theory and explicit constructions of emph{orthogonal designs} based on the seminal work of Bourgain and Gamburd (2011) on expansion in Lie groups.