We derive the bias, variance, covariance, and mean square error of the standard lag windowed correlogram estimator both with and without sample mean removal for complex white noise with an arbitrary mean. We find that the arbitrary mean introduces lag dependent covariance between different lags of the correlogram estimates in spite of the lack of covariance in white noise for non-zeros lags. We provide a heuristic rule for when the sample mean should be, and when it should not be, removed if the true mean is not known. The sampling properties derived here are useful is assesing the general statistical performance of autocovariance and autocorrelation estimators in different parameter regimes. Alternatively, the sampling properties could be used as bounds on the detection of a weak signal in general white noise.