We discuss the structure of the parton quasi-distributions (quasi-PDFs) $Q(y, P_3)$ outside the canonical $-1 leq y leq 1$ support region of the usual parton distribution functions (PDFs). Writing the $y^n$ moments of $Q(y, P_3)$ in terms of the combined $x^{n-2l} k_perp^{2l}$-moments of the transverse momentum distribution (TMD) ${cal F} (x,k_perp^2)$, we establish a connection between the large-$|y|$ behavior of $Q(y,P_3)$ and large-$k_perp^2$ behavior of ${cal F} (x,k_perp^2)$. In particular, we show that the $1/k_perp^2$ hard tail of TMDs in QCD results in a slowly decreasing $sim 1/|y|$ behavior of quasi-PDFs for large $|y|$ that produces infinite $y^n$ moments of $Q(y,P_3)$. We also relate the $sim 1/|y|$ terms with the $ln z_3^2$-singulariies of the Ioffe-time pseudo-distributions $mathfrak{M} ( u, z_3^2)$. Converting the operator product expansion for $mathfrak{M} ( u, z_3^2)$ into a matching relation between the quasi-PDF $Q(y,P_3)$ and the light-cone PDF $f(x, mu^2)$, we demonstrate that there is no contradiction between the infinite values of the $y^n$ moments of $Q(y,P_3)$ and finite values of the $x^n$ moments of $f(x, mu^2)$.