In this paper we give an explicit formula for the twisted Alexander polynomial of any torus link and show that it is a locally constant function on the $SL(2, mathbb C)$-character variety. We also discuss similar things for the higher dimensional twisted Alexander polynomial and the Reidemeister torsion.