The chiral susceptibility, or the first derivative of the chiral condensate with respect to the quark mass, is often used as a probe for the QCD phase transition since the chiral condensate is an order parameter of $SU(2)_L times SU(2)_R$ symmetry breaking. However, the chiral condensate also breaks the axial $U(1)$ symmetry, which is usually not paid attention to as it is already broken by anomaly. We investigate the susceptibilities in the scalar and pseudoscalar channels in order to quantify how much the axial $U(1)$ anomaly contributes to the chiral phase transition. Employing a chirally symmetric lattice Dirac operator, and its eigenmode decomposition, we separate the axial $U(1)$ breaking effects from others. Our result in two-flavor QCD indicates that the chiral susceptibility is dominated by the axial $U(1)$ anomaly at temperatures $Tgtrsim 190$ MeV after the quadratically divergent constant is subtracted.