The WALOP-South instrument will be mounted on the 1 m SAAO telescope in South Africa as part of the PASIPHAE program to carry out a linear imaging polarization survey of the Galactic polar regions in the optical band. Designed to achieve polarimetric sensitivity of $0.05~%$ across a $35times35$ arcminute field of view, it will be capable of measuring the Stokes parameters I, q and u in a single exposure in the SDSS-r broadband and narrowband filters between $0.5~{mu}m - 0.7~{mu}m$. For each measurement, four images of the full field corresponding to linear polarization angles of 0 deg, 45 deg, 90 deg and 135 deg in the instrument coordinate system will be created on four detectors from which the Stokes parameters can be found using differential photometry. In designing the optical system, major challenges included correcting for the dispersion introduced by large split angle Wollaston Prisms used as analysers as well as other aberrations from the entire field to obtain imaging quality PSF at the detector. We present the optical design of the WALOP-South instrument which overcomes these challenges and delivers near seeing limited PSFs for the entire field of view.