We show that a non-minimal coupling of electromagnetism with background torsion can produce birefringence of the electromagnetic waves. This birefringence gives rise to a B-mode polarization of the CMB. From the bounds on B-mode from WMAP and BOOMERanG data, one can put limits on the background torsion at $xi_{1}T_{1}=(-3.35 pm 2.65) times 10^{-22} GeV^{-1}$.