We present on-sky polarimetric observations with the Gemini Planet Imager (GPI) obtained at straight Cassegrain focus on the Gemini South 8-m telescope. Observations of polarimetric calibrator stars, ranging from nearly unpolarized to strongly polarized, enable determination of the combined telescope and instrumental polarization. We find the conversion of Stokes $I$ to linear and circular instrumental polarization in the instrument frame to be $I rightarrow (Q_{rm IP}, U_{rm IP}, P_{rm IP}, V_{rm IP}) = (-0.037 pm 0.010%, +0.4338 pm 0.0075%, 0.4354 pm 0.0075%, -6.64 pm 0.56%)$. Such precise measurement of instrumental polarization enables $sim 0.1%$ absolute accuracy in measurements of linear polarization, which together with GPIs high contrast will allow GPI to explore scattered light from circumstellar disk in unprecedented detail, conduct observations of a range of other astronomical bodies, and potentially even study polarized thermal emission from young exoplanets. Observations of unpolarized standard stars also let us quantify how well GPIs differential polarimetry mode can suppress the stellar PSF halo. We show that GPI polarimetry achieves cancellation of unpolarized starlight by factors of 100-200, reaching the photon noise limit for sensitivity to circumstellar scattered light for all but the smallest separations at which the calibration for instrumental polarization currently sets the limit.