We introduce a novel time-frequency quantum key distribution (TFQKD) scheme based on photon pairs entangled in these two conjugate degrees of freedom. The scheme uses spectral detection and phase modulation to enable measurements in the temporal basis by means of time-to-frequency conversion. This allows large-alphabet encoding to be implemented with realistic components. A general security analysis for TFQKD with binned measurements reveals a close connection with finite-dimensional QKD protocols and enables analysis of the effects of dark counts on the secure key size.