Monte Carlo simulation of gauge theories with a $theta$ term is known to be extremely difficult due to the sign problem. Recently there has been major progress in solving this problem based on the idea of complexifying dynamical variables. Here we consider the complex Langevin method (CLM), which is a promising approach for its low computational cost. The drawback of this method, however, is the existence of a condition that has to be met in order for the results to be correct. As a first step, we apply the method to 2D U(1) gauge theory on a torus with a $theta$ term, which can be solved analytically. We find that a naive implementation of the method fails because of the topological nature of the $theta$ term. In order to circumvent this problem, we simulate the same theory on a punctured torus, which is equivalent to the original model in the infinite volume limit for $ |theta| < pi$. Rather surprisingly, we find that the CLM works and reproduces the exact results for a punctured torus even at large $theta$, where the link variables near the puncture become very far from being unitary.