HR4796A hosts a well-studied debris disk with a long history due to its high fractional luminosity and favorable inclination lending itself well to both unresolved and resolved observations. We present new J- and K1-band images of the resolved debris disk HR4796A taken in the polarimetric mode of the Gemini Planet Imager (GPI). The polarized intensity features a strongly forward scattered brightness distribution and is undetected at the far side of the disk. The total intensity is detected at all scattering angles and also exhibits a strong forward scattering peak. We use a forward modelled geometric disk in order to extract geometric parameters, polarized fraction and total intensity scattering phase functions for these data as well as H-band data previously taken by GPI. We find the polarized phase function becomes increasingly more forward scattering as wavelength increases. We fit Mie and distribution of hollow spheres grain (DHS) models to the extracted functions. We find that while it is possible to describe generate a satisfactory model for the total intensity using a DHS model, but not with a Mie model. We find that no single grain population of DHS or Mie grains of arbitrary composition can simultaneously reproduce the polarized fraction and total intensity scattering phase functions, indicating the need for more sophisticated grain models.