Relativistic quantum molecular dynamics based on the relativistic mean field theory (RQMD.RMF) is extended by including momentum-dependent potential. The equation of state (EoS) dependence of the directed and the elliptic flow of protons in the beam energy range of $2.3 < sqrt{s_{NN}}< 20$ GeV is examined. It is found that the directed flow depends strongly on the optical potential at high energies,$sqrt{s_{NN}} > 3 $ GeV, where no information is available experimentally. The correlation between effective mass at saturation density and the optical potential is found: smaller values of effective mass require smaller strengths of the optical potential to describe the directed flow data.This correlation can also be seen in the beam energy dependence of the elliptic flow at $sqrt{s_{NN}}>3$ GeV, although its effect is rather weak. On the other hand, stiff EoS is required to describe the elliptic flow at lower energies.Experimental constraints on the optical potential from $pA$ collisions will provide important information on the EoS at high energies.The proton directed and the elliptic flow are well described in the RQMD.RMF model from $sqrt{s_{NN}}=2.3$ to 8.8 GeV. In contrast,to reproduce the collapse of the directed flow above 10 GeV, pressure has to be reduced, which indicates a softening of the EoS around $sqrt{s_{NN}} =10 $ GeV.