Disintegrating planets allow for the unique opportunity to study the composition of the interiors of small, hot, rocky exoplanets because the interior is evaporating and that material is condensing into dust, which is being blown away and then transiting the star. Their transit signal is dominated by dusty effluents forming a comet-like tail trailing the host planet (or leading it, in the case of K2-22b), making these good candidates for transmission spectroscopy. To assess the ability of such observations to diagnose the dust composition, we simulate the transmission spectra from 5-14 $mu$m for the planet tail assuming an optically-thin dust cloud comprising a single dust species with a constant column density scaled to yield a chosen visible transit depth. We find that silicate resonant features near 10 $mu$m can produce transit depths that are at least as large as those in the visible. For the average transit depth of 0.55% in the Kepler band for K2-22b, the features in the transmission spectra can be as large as 1%, which is detectable with the JWST MIRI low-resolution spectrograph in a single transit. The detectability of compositional features is easier with an average grain size of 1 $mu$m despite features being more prominent with smaller grain sizes. We find most features are still detectable for transit depths of ~0.3% in the visible range. If more disintegrating planets are found with future missions such as the space telescope TESS, follow-up observations with JWST can explore the range of planetary compositions.