Transient non-gaussian noise in gravitational wave detectors, commonly referred to as glitches, pose challenges for inference of the astrophysical properties of detected signals when the two are coincident in time. Current analyses aim towards modeling and subtracting the glitches from the data using a flexible, morphology-independent model in terms of sine-gaussian wavelets before the signal source properties are inferred using templates for the compact binary signal. We present a new analysis of gravitational wave data that contain both a signal and glitches by simultaneously modeling the compact binary signal in terms of templates and the instrumental glitches using sine-gaussian wavelets. The model for the glitches is generic and can thus be applied to a wide range of glitch morphologies without any special tuning. The simultaneous modeling of the astrophysical signal with templates allows us to efficiently separate the signal from the glitches, as we demonstrate using simulated signals injected around real O2 glitches in the two LIGO detectors. We show that our new proposed analysis can separate overlapping glitches and signals, estimate the compact binary parameters, and provide ready-to-use glitch-subtracted data for downstream inference analyses.