It was proposed that the $id_{x^2-y^2}$ density-wave state (DDW) may be responsible for the pseudogap behavior in the underdoped cuprates. Here we show that the admixture of a small $d_{xy}$ component to the DDW state breaks the symmetry between the counter-propagating orbital currents of the DDW state and, thus, violates the macroscopic time-reversal symmetry. This symmetry breaking results in a non-zero polar Kerr effect, which has recently been observed in the pseudogap phase.