This paper investigates the relations between modular graph forms, which are generalizations of the modular graph functions that were introduced in earlier papers motivated by the structure of the low energy expansion of genus-one Type II superstring amplitudes. These modular graph forms are multiple sums associated with decorated Feynman graphs on the world-sheet torus. The action of standard differential operators on these modular graph forms admits an algebraic representation on the decorations. First order differential operators are used to map general non-holomorphic modular graph functions to holomorphic modular forms. This map is used to provide proofs of the identities between modular graph functions for weight less than six conjectured in earlier work, by mapping these identities to relations between holomorphic modular forms which are proven by holomorphic methods. The map is further used to exhibit the structure of identities at arbitrary weight.