Based on the density functional theory with hybrid functional approach, we have studied the structural and thermodynamic stabilities of Cu2MSnX4 (M = Zn, Mg, and Ca; X = S and Se) alloy, and have further investigated the electronic and optical properties of stable Cu2MgSnS4 and Cu2MgSnSe4 phases. Thermal stability analysis indicates that Cu2MgSnS4 and Cu2MgSnSe4 are thermodynamically stable, while Cu2CaSnS4 and Cu2CaSnSe4 are unstable. The ground state configuration of the compound changes from kesterite into stannite structure when Zn atoms are substitued by larger Mg or Ca atoms. An energy separation between stannite and kesterite phase similar to that of CZTS is observed. Calculated electronic structures and optical properties suggest that Cu2MgSnS4 and Cu2MgSnSe4 can be efficient photovoltaic materials.