We present JHKs photometry of 10 Jovian and 4 Saturnian irregular satellites, taken with the Near-InfraRed Imager (NIRI) at the 8-m Gemini North Observatory on Mauna Kea, Hawaii. The observed objects have near-infrared colors consistent with C, P and D-type asteroids, although J XII Ananke and S IX Phoebe show weak indications of possible water features in the H filter. The four members of the Himalia-family have similar near-infrared colors, as do the two members of the Gallic family, S XX Paaliaq and S XXIX Siarnaq. From low resolution normalized reflectance spectra based on the broadband colors and covering 0.4 to 2.2 microns, the irregular satellites are identified as C-type (J VII Pasiphae, J VI Himalia and S IX Phoebe), P-type (J XII Ananke and J XVIII Themisto) and D-type (J IX Carme and J X Sinope), showing a diversity of origins of these objects.