We establish a general connection between ballistic and diffusive transport in systems where the ballistic contribution in canonical ensemble vanishes. A lower bound on the Green-Kubo diffusion constant is derived in terms of the curvature of the ideal transport coefficient, the Drude weight, with respect to the filling parameter. As an application, we explicitly determine the lower bound on the high temperature diffusion constant in the anisotropic spin 1/2 Heisenberg chain for anisotropy parameters $Delta geq 1$, thus settling the question whether the transport is sub-diffusive or not. Addi- tionally, the lower bound is shown to saturate the diffusion constant for a certain classical integrable model.