We employ time-resolved resonant x-ray diffraction to study the melting of charge order and the associated insulator-metal transition in the doped manganite Pr$_{0.5}$Ca$_{0.5}$MnO$_3$ after resonant excitation of a high-frequency infrared-active lattice mode. We find that the charge order reduces promptly and highly nonlinearly as function of excitation fluence. Density functional theory calculations suggest that direct anharmonic coupling between the excited lattice mode and the electronic structure drive these dynamics, highlighting a new avenue of nonlinear phonon control.