Powerful constraints on theories can already be inferred from existing CMB anisotropy data. But performing an exact analysis of available data is a complicated task and may become prohibitively so for upcoming experiments with gtrsim10^4 pixels. We present a method for approximating the likelihood that takes power spectrum constraints, e.g., ``band-powers, as inputs. We identify a bias which results if one approximates the probability distribution of the band-power errors as Gaussian---as is the usual practice. This bias can be eliminated by using specific approximations to the non-Gaussian form for the distribution specified by three parameters (the maximum likelihood or mode, curvature or variance, and a third quantity). We advocate the calculation of this third quantity by experimenters, to be presented along with the maximum-likelihood band-power and variance. We use this non-Gaussian form to estimate the power spectrum of the CMB in eleven bands from multipole moment ell = 2 (the quadrupole) to ell=3000 from all published band-power data. We investigate the robustness of our power spectrum estimate to changes in these approximations as well as to selective editing of the data.