We investigate the polarization of Compton scattered X-rays from relativistic jets in active galactic nuclei using Monte Carlo simulations. We consider three scenarios: scattering of photons from an accretion disk, scattering of cosmic microwave background (CMB) photons, and synchrotron self-Comptonization (SSC) within the jet. For Comptonization of thermal disk photons or CMB photons the maximum linear polarization attained is slightly over 20% at viewing angles close to 90 degrees. The value decreases with the viewing inclination. For SSC, the maximum value may exceed 80%. The angle dependence is complicated, and it varies with the photon injection sites. Our study demonstrates that X-ray polarization, in addition to multi-wavelength spectra, can distinguish certain models for emission and particle acceleration in relativistic jets.