We study the ultrafast phonon response of mixed-valence perovskite Cs$_2$Au$_2$I$_6$ using pump-probe spectroscopy under high-pressure in a diamond anvil cell. We observed a remarkable softening and broadening of the Au - I stretching phonon mode wit
h both applied pressure and photoexcitation. Using a double-pump scheme we measured a lifetime of the charge transfer excitation into single valence Au$^{2+}$ of less than 4 ps, which is an indication of the local character of the Au$^{2+}$ excitation. Furthermore, the strong similarity between the pressure and fluence dependence of the phonon softening shows that the inter-valence charge transfer plays an important role in the structural transition.
