In this paper, we study the semilinear wave equations with the inverse-square potential. By transferring the original equation to a fractional dimensional wave equation and analyzing the properties of its fundamental solution, we establish a long-time existence result, for sufficiently small, spherically symmetric initial data. Together with the previously known blow-up result, we determine the critical exponent which divides the global existence and finite time blow-up. Moreover, the sharp lower bounds of the lifespan are obtained, except for certain borderline case. In addition, our technology allows us to handle an extreme case for the potential, which has hardly been discussed in literature.