We demonstrate a method for reconstructing the weak lensing potential from the Lyman-$alpha$ forest data. We derive an optimal estimator for the lensing potential on the sky based on the correlation between pixels in real space. This method effectively deals with irregularly spaced data, holes in the survey, missing data and inhomogeneous noise. We demonstrate an implementation of the method with simulated spectra and weak lensing. It is shown that with a source density of $>sim 0.5$ per square arcminutes and $sim 200$ pixels in each spectrum ($lambda / Deltalambda = 1300$) the lensing potential can be reconstructed with high fidelity if the relative absorption in the spectral pixels is signal dominated. When noise dominates the measurement of the absorption in each pixel the noise in the lensing potential is higher, but for reasonable numbers of sources and noise levels and a high fidelity map the lensing potential is obtainable. The lensing estimator could also be applied to lensing of the Cosmic Microwave Background (CMB), 21 cm intensity mapping (IM) or any case in which the correlation function of the source can be accurately estimated.