The Mid-Infrared Instrument MIRI on-board the James Webb Space Telescope uses three Si:As impurity band conduction detector arrays. MIRI medium resolution spectroscopic measurements (R$sim$3500-1500) in the 5~$mu m$ to 28~$mu m$ wavelength range show a 10-30% modulation of the spectral baseline; coherent reflections of infrared light within the Si:As detector arrays result in fringing. We quantify the shape and impact of fringes on spectra of optical sources observed with MIRI during ground testing and develop an optical model to simulate the observed modulation. We use our optical model in conjunction with the MIRI spectroscopic data to show that the properties of the buried contact inside the MIRI Si:As detector have a significant effect on the fringing behavior.