A new Chandra observation of SNR 0506-68.0 (also called N23) reveals a complex, highly structured morphology in the low energy X-ray band and an isolated compact central object in the high energy band. Spectral analysis indicates that the X-ray emission overall is dominated by thermal gas whose composition is consistent with swept-up ambient material. There is a strong gradient in ambient density across the diameter of the remnant. Toward the southeast, near a prominent star cluster, the emitting density is 10 - 23 cm^{-3} while toward the northwest it has dropped to a value of only 1 cm^{-3}. The total extent of the X-ray remnant is 100 by 120 (24 pc x 29 pc for a distance of 50 kpc), somewhat larger than previously known. The remnants age is estimated to be ~4600 yr. One part of the remnant shows evidence for enhanced O, Ne, and perhaps Mg abundances, which is interpreted as evidence for ejecta from a massive star core collapse supernova. The compact central object has a luminosity of a few times 10^{33} ergs/s and no obvious radio or optical counterpart. It does not show an extended nebula or pulsed emission as expected from a young energetic pulsar, but resembles the compact central objects seen in other core collapse SNe, such as Cas A.