We propose fast phase-gates of single nuclear spins interacting with single electron spins. The gate operation utilizes geometric phase shifts of the electron spin induced by fast/slow rotating fields; the path difference depending on nuclear spin states enables nuclear phase shifts. The gate time is inversely proportional to the frequency of the slow rotating field. As an example, we use nitrogen-vacancy centers in diamond, and show the phase-gate time orders of magnitude shorter than previously reported. We also show the robustness of the gate against decoherence and systematic errors.