We report on the results of deep X-ray follow-up observations of four unidentified Fermi/LAT gamma-ray sources at high Galactic latitudes using Suzaku. The studied objects were detected with high significance during the first 3 months of Fermi/LAT operation, and subsequently better localized in the Fermi/LAT 1 year catalog (1FGL). Possible associations with pulsars and active galaxies have subsequently been discussed, and our observations provide an important contribution to this debate. In particular, an X-ray point source was found within the 95% confidence error circle of 1FGL J1231.1-1410. X-ray spectrum is well-fitted by a blackbody with an additional power-law. This supports the recently claimed identification of this source with a millisecond pulsar (MSP) PSR J1231-1411. Concerning 1FGL J1311.7-3429, two X-ray sources were found within the LAT error circle. Even though the X-ray spectral and variability properties were accessed, their nature and relationship with the gamma-ray source remain uncertain. We found several weak X-ray sources in the field of 1FGL J1333.2+5056, one coinciding with CLASS J1333+5057. We argue the available data are consistent with the association between these two objects. Finally, we have detected an X-ray source in the vicinity of 1FGL J2017.3+0603. This object was recently suggested to be associated with a newly discovered MSP PSR J2017+0603, because of the spatial-coincidence and the gamma-ray pulse detection. We have only detected the X-ray counterpart of the CLASS J2017+0603, while we determined an X-ray flux upper limit at the pulsar position. All in all, our studies indicate while a significant fraction of unidentified high Galactic latitude gamma-ray sources is related to the pulsar and blazar phenomena, associations with other classes of astrophysical objects are still valid options.