We present further evidence for a dual conformal symmetry in the four-gluon planar scattering amplitude in N=4 SYM. We show that all the momentum integrals appearing in the perturbative on-shell calculations up to five loops are dual to true conformal integrals, well defined off shell. Assuming that the complete off-shell amplitude has this dual conformal symmetry and using the basic properties of factorization of infrared divergences, we derive the special form of the finite remainder previously found at weak coupling and recently reproduced at strong coupling by AdS/CFT. We show that the same finite term appears in a weak coupling calculation of a Wilson loop whose contour consists of four light-like segments associated with the gluon momenta. We also demonstrate that, due to the special form of the finite remainder, the asymptotic Regge limit of the four-gluon amplitude coincides with the exact expression evaluated for arbitrary values of the Mandelstam variables.