We systematically investigate the dependence of the temperature of cold cesium atoms of polarization gradient cooling (PGC) in optical molasses on experimental parameters, which contain changing modes of cooling laser, PGC interaction time, cooling laser frequency and its intensity. The SR mode of cooling laser, that means the cooling laser frequency is changed with step mode and cooling laser intensity is changed with ramp mode, is found to be the best for PGC comparing with other SS, RS, and RR modes. We introduce a statistical explanation and an exponential decay function to explain the variation of cold atomic temperature on time. The heating effect is observed when the cooling laser intensity is lower than the saturation intensity of cold atoms. After optimization, the lowest temperature of cold cesium atoms is observed to be about 4uK with the number of 2x10^9, a density of 1x10^11/cm^3 and the phase space density of 4.4x10^(-5). The optimization process and analysis of controllable experimental parameters are also meaningful for other cold atomic systems.