We conducted near-infrared (JHKs) imaging polarimetry toward the infrared dark cloud (IRDC) M17 SWex, including almost all of the IRDC filaments as well as its outskirts, with the polarimeter SIRPOL on the IRSF 1.4 m telescope. We revealed the magnetic fields of M17 SWex with our polarization-detected sources that were selected by some criteria based on their near-IR colors and the column densities toward them, which were derived from the Herschel data. The selected sources indicate not only that the ordered magnetic field is perpendicular to the cloud elongation as a whole, but also that at both ends of the elongated cloud the magnetic field appears to bent toward its central part, i.e., large-scale hourglass-shaped magnetic field perpendicular to the cloud elongation. In addition to this general trend, the elongations of the filamentary subregions within the dense parts of the cloud appear to be mostly perpendicular to their local magnetic fields, while the magnetic fields of the outskirts appear to follow the thin filaments that protrude from the dense parts. The magnetic strengths were estimated to be ~70-300 microG in the subregions, of which lengths and average number densities are ~3-9 pc and ~2-7x10^3 cm^{-3}, respectively, by the Davis-Chandrasekhar-Fermi method with the angular dispersion of our polarization data and the velocity dispersion derived from the C^{18}O (J=1-0) data obtained by the Nobeyama 45 m telescope. These field configurations and our magnetic stability analysis of the subregions imply that the magnetic field have controlled the formation/evolution of the M17 SWex cloud.