The quantum theory of a massless spin two particle is strongly constrained by diffeomorphism invariance, which is in turn implied by unitarity. We explicitly exhibit the space-time diffeomorphism algebra of string theory, realizing it in terms of world sheet vertex operators. Viewing diffeomorphisms as field redefinitions in the two-dimensional conformal field theory renders the calculation of their algebra straightforward. Next, we generalize the analysis to combinations of space-time anti-symmetric tensor gauge transformations and diffeomorphisms. We also point out a left-right split of the algebra combined with a twist that reproduces the C-bracket of double field theory. We further compare our derivation to an analysis in terms of marginal deformations as well as vertex operator algebras.