This paper is concerned with the mathematical analysis of the time-domain electromagnetic scattering problem in an infinite rectangular waveguide. A transparent boundary condition is developed to reformulate the problem into an equivalent initial boundary value problem in a bounded domain. The well-posedness and stability are obtained for the reduced problem. The perfectly matched layer method is studied to truncate the waveguide. It is shown that the truncated problem attains a unique solution. Moreover, an explicit error estimate is given between the solutions of the original scattering problem and the truncated problem. Based on the estimate, the stability and exponential convergence are established for the truncated problem. The optimal bound is achieved for the error with explicit dependence on the parameters of the perfectly matched layer.