High-quality rare-earth-ion (REI) doped materials are a prerequisite for many applications such as quantum memories, ultra-high-resolution optical spectrum analyzers and information processing. Compared to bulk materials, REI doped powders offer low-cost fabrication and a greater range of accessible material systems. Here we show that crystal properties, such as nuclear spin lifetime, are strongly affected by mechanical treatment, and that spectral hole burning can serve as a sensitive method to characterize the quality of REI doped powders. We focus on the specific case of thulium doped Y$_2$Al$_5$O$_{12}$ (Tm:YAG). Different methods for obtaining the powders are compared and the influence of annealing on the spectroscopic quality of powders is investigated on a few examples. We conclude that annealing can reverse some detrimental effects of powder fabrication and, in certain cases, the properties of the bulk material can be reached. Our results may be applicable to other impurities and other crystals, including color centers in nano-structured diamond.