Frequency translation of single photons while preserving their quantum characteristics is an important technology for flexible networking of photonic quantum communication systems. Here we demonstrate a flexible scheme to interface different-color photons using an optical single sideband (OSSB) modulator. By changing the radio-frequency signal that drives the modulators, we can easily shift and precisely tune the frequency of single photons. Using the OSSB modulator, we successfully erased the frequency distinguishability of non-degenerated photon pairs to obtain the Hong-Ou-Mandel interference with a visibility exceeding 90%. We also demonstrated that the level of distinguishability can be precisely controlled by the OSSB modulator. We expect that the OSSB modulator will provide a simple and flexible photonic interface for realizing advanced quantum information systems.