In this paper, we compute the motive of the character variety of representations of the fundamental group of the complement of an arbitrary torus knot into $SL_4(k)$, for any algebraically closed field $k$. For that purpose, we introduce a stratification of the variety in terms of the type of a canonical filtration attached to any representation. This allows us to reduce the computation of the motive to a combinatorial problem.