We study the generalisations of the Craps-Sethi-Verlinde matrix big bang model to curved, in particular plane wave, space-times, beginning with a careful discussion of the DLCQ procedure. Singular homogeneous plane waves are ideal toy-models of realistic space-time singularities since they have been shown to arise universally as their Penrose limits, and we emphasise the role played by the symmetries of these plane waves in implementing the flat space Seiberg-Sen DLCQ prescription for these curved backgrounds. We then analyse various aspects of the resulting matrix string Yang-Mills theories, such as the relation between strong coupling space-time singularities and world-sheet tachyonic mass terms. In order to have concrete examples at hand, in an appendix we determine and analyse the IIA singular homogeneous plane wave - null dilaton backgrounds.