The Large Area Telescope (LAT) on Fermi has detected ~150 gamma-ray pulsars, about a third of which were discovered in blind searches of the $gamma$-ray data. Because the angular resolution of the LAT is relatively poor and blind searches for pulsars (especially millisecond pulsars, MSPs) are very sensitive to an error in the position, one must typically scan large numbers of locations. Identifying plausible X-ray counterparts of a putative pulsar drastically reduces the number of trials, thus improving the sensitivity of pulsar blind searches with the LAT. I discuss our ongoing program of Swift, XMM-Newton, and Chandra observations of LAT unassociated sources in the context of our blind searches for gamma-ray pulsars.