Here we describe the results of an atomic resolution study of the structure and composition of both the interior of the grains, and the grain boundaries in polycrystalline MgB2. We find that there is no oxygen within the bulk of the grains but significant oxygen enrichment at the grain boundaries. The majority of grain boundaries contain BOx phases smaller than the coherence length, while others contain larger areas of MgO sandwiched between BOx layers. Such results naturally explain the differences in connectivity between the grains observed by other techniques.