A new detection method for gravitational waves (GWs) with ultra-low frequencies ($f_{rm GW} lesssim 10^{-10}~{rm Hz}$), which is much lower than the range of pulsar timing arrays (PTAs), was proposed in Yonemaru et al. (2016). This method utilizes the statistical properties of spin-down rates of milli-second pulsars (MSPs) and the sensitivity was evaluated in Yonemaru et al. (2018). There, some simplifying assumptions, such as neglect of the pulsar term and spatially uniform distribution of MSPs, were adopted and the sensitivity on the time derivative of GW amplitude was estimated to be $10^{-19}~{rm s}^{-1}$ independent of the direction, polarization and frequency of GWs. In this paper, extending the previous analysis, realistic simulations are performed to evaluate the sensitivity more reasonably. We adopt a model of 3-dimensional pulsar distribution in our Galaxy and take the pulsar term into account. As a result, we obtain expected sensitivity as a function of the direction, polarization and frequency of GWs. The dependence on GW frequency is particularly significant and the sensitivity becomes worse by a few orders for $< 10^{-12}~{rm Hz}$ compared to the previous estimates.