In this work, perovskite solar cells (PSCs) with different transport layers were fabricated to understand the hysteresis phenomenon under a series of scan rates. The experimental results show that the hysteresis phenomenon would be affected by the dielectric constant of transport layers and scan rate significantly. To explain this, a modified Poisson and drift-diffusion solver coupled with a fully time-dependent ion migration model is developed to analyze how the ion migration affects the performance and hysteresis of PSCs. The simulation model was optimized for carrier transportation of organic materials, which can simulate the organic transport layer correctly without using heavy doping in simulating the organic transport layer. The modeling results show that the most crucial factor in the hysteresis behavior is the built-in electric field of the perovskite. The non-linear hysteresis curves are demonstrated under different scan rates, and the mechanism of the hysteresis behavior is explained. The findings reveal why the change in hysteresis degree with scan rate is Gaussian shaped rather than monotonic. Additionally, other factors contributing to the degree of hysteresis are determined to be the degree of degradation in the perovskite material, the quality of the perovskite crystal, and the materials of the transport layer, which corresponds to the total ion density, carrier lifetime of perovskite, and the dielectric constant of the transport layer, respectively. Finally, it was found that the dielectric constant of the transport layer is a key factor affecting hysteresis in perovskite solar cells; a lower dielectric constant corresponds to a higher electric field of the transport layer. Hence, if the electric field of the perovskite material is small, the degree of hysteresis is small and vice versa.