We have investigated the ac susceptibility of the spin triplet superconductor Sr$_2$RuO$_4$ as a function of magnetic field in various directions at temperatures down to 60 mK. We have focused on the in-plane field configuration (polar angle $theta simeq 90^{circ}$), which is a prerequisite for inducing multiple superconducting phases in Sr$_2$RuO$_4$. We have found that the previous attribution of a pronounced feature in the ac susceptibility to the second superconducting transition itself is not in accord with recent measurements of the thermal conductivity or of the specific heat. We propose that the pronounced feature is a consequence of additional involvement of vortex pinning originating from the second superconducting transition.