Self-interacting Dark Matter (SIDM) is a leading candidate proposed to solve discrepancies between predictions of the prevailing cold dark matter theory and observations of galaxies. Many SIDM models predict the existence of a light force carrier that mediate strong dark matter self-interactions. If the mediator couples to the standard model particles, it could produce characteristic signals in dark matter direct detection experiments. We report searches for SIDM models with a light mediator using the full dataset of the PandaX-II experiment, based on a total exposure of 132 tonne-days. No significant excess over background is found, and our likelihood analysis leads to a strong upper limit on the dark matter-nucleon coupling strength. We further combine the PandaX-II constraints and those from observations of the light element abundances in the early universe, and show that direct detection and cosmological probes can provide complementary constraints on dark matter models with a light mediator.