We demonstrate an attack on a clock synchronization protocol that attempts to detect tampering of the synchronization channel using polarization-entangled photon pairs. The protocol relies on a symmetrical channel, where propagation delays do not depend on propagation direction, for correctly deducing the offset between clocks -- a condition that could be manipulated with optical circulators, which rely on static magnetic fields to break the reciprocity of propagating electromagnetic fields. Despite the polarization transformation induced within a set of circulators, our attack creates an error in time synchronization while successfully evading detection.