One of the difficulties with performing polarization analysis is that the mean polarization fraction of sub-divided data sets is larger than the polarization fraction for the integrated measurement. The resulting bias is one of the properties of the generating distribution discussed in this work. The limitations of Gaussian approximations in standard analysis based on Stokes parameters for estimating polarization parameters and their uncertainties are explored by comparing with a Bayesian analysis. Different signal-to-background scenarios are considered making the analysis relevant for a large variety of observations. The effect of uncertainty on the modulation factor is also shown, since it can have a large impact on the performance of gamma-ray burst polarimeters. Results are related to the minimum detectable polarization (MDP), a common figure of merit, making them easily applicable to any X-ray polarimeter.