Though theoretically expected, the charge exchange emission from galaxy clusters has not yet been confidently detected. Accumulating hints were reported recently, including a rather marginal detection with the Hitomi data of the Perseus cluster. As suggested in Gu et al. (2015), a detection of charge exchange line emission from galaxy clusters would not only impact the interpretation of the newly-discovered 3.5 keV line, but also open up a new research topic on the interaction between hot and cold matter in clusters. We aim to perform the most systematic search for the O VIII charge exchange line in cluster spectra using the RGS on board XMM. We introduce a sample of 21 clusters observed with the RGS. The dominating thermal plasma emission is modeled and subtracted with a two-temperature CIE component, and the residuals are stacked for the line search. The systematic uncertainties in the fits are quantified by refitting the spectra with a varying continuum and line broadening. By the residual stacking, we do find a hint of a line-like feature at 14.82 A, the characteristic wavelength expected for oxygen charge exchange. This feature has a marginal significance of 2.8 sigma, and the average equivalent width is 2.5E-4 keV. We further demonstrate that the putative feature can be hardly affected by the systematic errors from continuum modelling and instrumental effects, or the atomic uncertainties of the neighbouring thermal lines. Assuming a realistic temperature and abundance pattern, the physical model implied by the possible oxygen line agrees well with the theoretical model proposed previously to explain the reported 3.5 keV line. If the charge exchange source indeed exists, we would expect that the oxygen abundance is potentially overestimated by 8-22% in previous X-ray measurements which assumed pure thermal lines.