Diffusion tensor imaging (DTI) is a popular magnetic resonance imaging technique used to characterize microstructural changes in the brain. DTI studies quantify the diffusion of water molecules in a voxel using an estimated 3x3 symmetric positive definite diffusion tensor matrix. Statistical analysis of DTI data is challenging because the data are positive definite matrices. Matrix-variate information is often summarized by a univariate quantity, such as the fractional anisotropy (FA), leading to a loss of information. Furthermore, DTI analyses often ignore the spatial association of neighboring voxels, which can lead to imprecise estimates. Although the spatial modeling literature is abundant, modeling spatially dependent positive definite matrices is challenging. To mitigate these issues, we propose a matrix-variate Bayesian semiparametric mixture model, where the positive definite matrices are distributed as a mixture of inverse Wishart distributions with the spatial dependence captured by a Markov model for the mixture component labels. Conjugacy and the double Metropolis-Hastings algorithm result in fast and elegant Bayesian computing. Our simulation study shows that the proposed method is more powerful than non-spatial methods. We also apply the proposed method to investigate the effect of cocaine use on brain structure. The contribution of our work is to provide a novel statistical inference tool for DTI analysis by extending spatial statistics to matrix-variate data.