Single top quark production cross sections at hadron colliders are traditionally used to extract the modulus of the $V_{tb}$ element of the Cabibbo-Kobayashi-Maskawa matrix under the following assumption: $|V_{tb}| gg |V_{td}|, |V_{ts}|$. For the first time, direct limits on $|V_{td}|$ and $|V_{ts}|$ are obtained using experimental data without the assumption of the unitarity of the CKM matrix. Limits on the $|V_{td}|$, $|V_{ts}|$ and $|V_{tb}|$ are extracted from differential measurements of single top quark cross sections in $t$-channel as a function of the rapidity and transverse momentum of the top quark and the light jet recoiling against the top quark. We have shown that the pseudorapidity of the forward jet in the single top production is one of the most powerful observables for discriminating between the $|V_{td}|$ and $|V_{tb}|$ events. We perform a global fit of top quark related CKM elements to experimental data from the LHC Runs I and II and Tevatron. Experimental data include inclusive and differential single top cross sections in $t$-channel, inclusive tW production cross section, and top quark branching ratio to b quark and W boson. We present bounds on $|V_{tb}|$, $|V_{ts}|$ and $|V_{td}|$ using current data and project the results for future LHC data sets corresponding to luminosities of 300 and 3000 fb.