The Stochastic Gravitational Wave Background (SGWB) is expected to be a key observable for Gravitational Wave (GW) interferometry. Its detection will open a new window on early universe cosmology and on the astrophysics of compact objects. Using a Boltzmann approach, we study the angular anisotropies of the GW energy density, which is an important tool to disentangle the different cosmological and astrophysical contributions to the SGWB. Anisotropies in the cosmological background are imprinted both at its production, and by GW propagation through the large-scale scalar and tensor perturbations of the universe. The first contribution is not present in the Cosmic Microwave Background (CMB) radiation (as the universe is not transparent to photons before recombination), causing an order one dependence of the anisotropies on frequency. Moreover, we provide a new method to characterize the cosmological SGWB through its possible deviation from a Gaussian statistics. In particular, the SGWB will become a new probe of the primordial non-Gaussianity of the large-scale cosmological perturbations.