Gaussian double Markovian models consist of covariance matrices constrained by a pair of graphs specifying zeros simultaneously in the covariance matrix and its inverse. We study the semi-algebraic geometry of these models, in particular their dimension, smoothness and connectedness. Results on their vanishing ideals and conditional independence ideals are also included, and we put them into the general framework of conditional independence models. We end with several open questions and conjectures.