We consider solutions of the matrix KP hierarchy that are trigonometric functions of the first hierarchical time $t_1=x$ and establish the correspondence with the spin generalization of the trigonometric Calogero-Moser system on the level of hierarchies. Namely, the evolution of poles $x_i$ and matrix residues at the poles $a_i^{alpha}b_i^{beta}$ of the solutions with respect to the $k$-th hierarchical time of the matrix KP hierarchy is shown to be given by the Hamiltonian flow with the Hamiltonian which is a linear combination of the first $k$ higher Hamiltonians of the spin trigonometric Calogero-Moser system with coordinates $x_i$ and with spin degrees of freedom $a_i^{alpha}, , b_i^{beta}$. By considering evolution of poles according to the discrete time matrix KP hierarchy we also introduce the integrable discrete time version of the trigonometric spin Calogero-Moser system.