In earlier work we studied features of non-holomorphic modular functions associated with Feynman graphs for a conformal scalar field theory on a two-dimensional torus with zero external momenta at all vertices. Such functions, which we will refer to as modular graph functions, arise, for example, in the low energy expansion of genus-one Type II superstring amplitudes. We here introduce a class of single-valued elliptic multiple polylogarithms, which are defined as elliptic functions associated with Feynman graphs with vanishing external momenta at all but two vertices. These functions depend on a coordinate, $zeta$, on the elliptic curve and reduce to modular graph functions when $zeta$ is set equal to $1$. We demonstrate that these single-valued elliptic multiple polylogarithms are linear combinations of multiple polylogarithms, and that modular graph functions are sums of single-valued elliptic multiple polylogarithms evaluated at the identity of the elliptic curve, in both cases with rational coefficients. This insight suggests the many interrelations between modular graph functions (a few of which were established in earlier papers) may be obtained as a consequence of identities involving multiple polylogarithms, and explains an earlier observation that the coefficients of the Laurent polynomial at the cusp are given by rational numbers times single-valued multiple zeta values.