The dual harmonic system has been widely used in high intensity proton synchrotrons to suppress the space charge effect, as well as reduce the beam loss. To investigate the longitudinal beam dynamics in the dual rf system, the potential well, the sub-buckets in the bunch and the multi-solutions of the phase equation have been studied theoretically. Based on these theoretical studis, the optimization of bunching factor and rf voltage waveform are made for the dual harmonic rf system in the upgrade phase of the CSNS/RCS. In the optimization process, the simulation with space charge effect is done by using a newly developed code C-SCSIM.