The polarization entanglement photon pairs generated from the biexciton cascade decay in a single semiconductor quantum dot is corrupted by the position-dependent (time-dependent) phase difference of the two polarization mode due to the fine structure splitting. We show that, by taking voltage ramping to an electro-optic modulator, such phase-difference can be removed. In our first proposed set-up, two photons are sent to two separate Pockels cell under reverse voltage ramping, as a result, the position-dependent phase difference between the two polarization mode is removed in the outcome state. In our second proposed set-up, the polarization of the first photon is flipped and then both photons fly into the same Pokels cell. Since we only need to separate the two photons rather than separate the two polarization modes, our schemes are robust with respect to fluctuations of the optical paths.