We report active control of the friction force at the contact between a nanoscale asperity and a La$_{0.55}$Ca$_{0.45}$MnO$_3$ (LCMO) thin film using electric fields. We use friction force microscopy under ultrahigh vacuum conditions to measure the friction force as we change the film resistive state by electric field-induced resistive switching. Friction forces are high in the insulating state and clearly change to lower values when the probed local region is switched to the conducting state. Upon switching back to an insulating state, the friction forces increase again. Thus, we demonstrate active control of friction without having to change the contact temperature or pressure. By comparing with measurements of friction at the metal-to-insulator transition and with the effect of applied voltage on adhesion, we rule out electronic excitations, electrostatic forces and changes in contact area as the reasons for the effect of resistive switching on friction. Instead, we argue that friction is limited by phonon relaxation times which are strongly coupled to the electronic degrees of freedom through distortions of the MnO6 octahedra. The concept of controlling friction forces by electric fields should be applicable to any materials where the field produces strong changes in phonon lifetimes.