We report on detailed structural, electronic and magnetic studies of GdMn$_{1-x}$Cr$_x$O$_3$ for Cr doping levels 0 $le$ $x$ $le$ 1. In the solid solutions, the Jahn-Teller distortion associated with Mn$^{3+}$ ions gives rise to major changes in the ${bc}$-plane sub-lattice and also the effective orbital ordering in the ${ab}$-plane, which persist up to the compositions $x$ $sim$ 0.35. These distinct features in the lattice and orbital degrees of freedom are also correlated with $bc$-plane anisotropy of the local Gd environment. A gradual evolution of electronic states with doping is also clearly seen in O $K$-edge x-ray absorption spectra. Evidence of magnetization reversal in field-cooled-cooling mode for $x$ $ge$ 0.35 coinciding the Jahn-Teller crossover, suggests a close correlation between magnetic interaction and structural distortion. These observations indicate a strong entanglement between lattice, spin, electronic and orbital degrees of freedom. The nonmonotonic variation of remnant magnetization can be explained by doping induced modification of magnetic interactions. Density functional theory calculations are consistent with a layer-by-layer type doping with ferromagnetic (antiferomagnetic) coupling between Mn (Cr) ions for intermediate compound ($x$ = 0.5), which is distinct from that observed for the end members GMnO$_3$ and GdCrO$_3$.