Emission of fullerenes in their infrared vibrational bands has been detected in space near hot stars. The proposed attribution of the diffuse interstellar bands at 9577 and 9632 AA to electronic transitions of the buckminsterfullerene cation (i.e. C$_{60}^+$ ) was recently supported by new laboratory data, confirming the presence of this species in the diffuse interstellar medium (ISM). In this letter, we present the detection, also in the diffuse ISM, of the 17.4 and 18.9 $mu$m emission bands commonly attributed to vibrational bands of neutral C$_{60}$. According to classical models that compute the charge state of large molecules in space, C$_{60}$ is expected to be mostly neutral in the diffuse ISM. This is in agreement with the abundances of diffuse C$_{60}$ we derive here from observations.