In this paper, we introduce a multiscale framework based on adaptive edge basis functions to solve second-order linear elliptic PDEs with rough coefficients. One of the main results is that we prove the proposed multiscale method achieves nearly exponential convergence in the approximation error with respect to the computational degrees of freedom. Our strategy is to perform an energy orthogonal decomposition of the solution space into a coarse scale component comprising $a$-harmonic functions in each element of the mesh, and a fine scale component named the bubble part that can be computed locally and efficiently. The coarse scale component depends entirely on function values on edges. Our approximation on each edge is made in the Lions-Magenes space $H_{00}^{1/2}(e)$, which we will demonstrate to be a natural and powerful choice. We construct edge basis functions using local oversampling and singular value decomposition. When local information of the right-hand side is adaptively incorporated into the edge basis functions, we prove a nearly exponential convergence rate of the approximation error. Numerical experiments validate and extend our theoretical analysis; in particular, we observe no obvious degradation in accuracy for high-contrast media problems.