Fiber-based remote comparison of $^{87}$Sr lattice clocks in 24 km distant laboratories is demonstrated. The instability of the comparison reaches $5times10^{-16}$ over an averaging time of 1000 s, which is two orders of magnitude shorter than that of conventional satellite links and is limited by the instabilities of the optical clocks. By correcting the systematic shifts that are predominated by the differential gravitational redshift, the residual fractional difference is found to be $(1.0pm7.3)times10^{-16}$, confirming the coincidence between the two clocks. The accurate and speedy comparison of distant optical clocks paves the way for a future optical redefinition of the second.