Based on the analysis of the measurement data of angle-resolved photoemission spectroscopy (ARPES) and optics, we show that the charge transfer gap is significantly smaller than the optical one and is reduced by doping in electron doped cuprate superconductors. This leads to a strong charge fluctuation between the Zhang-Rice singlet and the upper Hubbard bands. The basic model for describing this system is a hybridized two-band $t$-$J$ model. In the symmetric limit where the corresponding intra- and inter-band hopping integrals are equal to each other, this two-band model is equivalent to the Hubbard model with an antiferromagnetic exchange interaction (i.e. the $t$-$U$-$J$ model). The mean-field result of the $t$-$U$-$J$ model gives a good account for the doping evolution of the Fermi surface and the staggered magnetization.