The hydrogen and helium accreted by X-ray bursting neutron stars is periodically consumed in runaway thermonuclear reactions that cause the entire surface to glow brightly in X-rays for a few seconds. With models of the emission, the mass and radius of the neutron star can be inferred from the observations. By simultaneously probing neutron star masses and radii, X-ray bursts are one of the strongest diagnostics of the nature of matter at extremely high densities. Accurate determinations of these parameters are difficult, however, due to the highly non-ideal nature of the atmospheres where X-ray bursts occur. Observations from X-ray telescopes such as RXTE and NuStar can potentially place strong constraints on nuclear matter once uncertainties in atmosphere models have been reduced. Here we discuss current progress on modeling atmospheres of X-ray bursting neutron stars and some of the challenges still to be overcome.