Both hydrodynamics-based models and a multi-phase transport (AMPT) model can reproduce the mass splitting of azimuthal anisotropy ($v_n$) at low transverse momentum ($p_{perp}$) as observed in heavy ion collisions. In the AMPT model, however, $v_n$ is mainly generated by the parton escape mechanism, not by the hydrodynamic flow. In this study we provide detailed results on the mass splitting of $v_n$ in this transport model, including $v_2$ and $v_3$ of various hadron species in d+Au and Au+Au collisions at the Relativistic Heavy Ion Collider and p+Pb collisions at the Large Hadron Collider. We show that the mass splitting of hadron $v_2$ and $v_3$ in AMPT first arises from the kinematics in the quark coalescence hadronization process, and then, more dominantly, comes from hadronic rescatterings, even though the contribution from the latter to the overall charged hadron $v_n$ is small. We further show that there is no qualitative difference between heavy ion collisions and small-system collisions or between elliptic ($v_2$) and triangular ($v_3$) anisotropies. Our studies thus demonstrate that the mass splitting of $v_2$ and $v_3$ at low-$p_{perp}$ is not a unique signature of hydrodynamic collective flow but can be the interplay of several physics effects.