Quantum computing in terms of geometric phases, i.e. Berry or Aharonov-Anandan phases, is fault-tolerant to a certain degree. We examine its implementation based on Zeeman coupling with a rotating field and isotropic Heisenberg interaction, which describe NMR and can also be realized in quantum dots and cold atoms. Using a novel physical representation of the qubit basis states, we construct $pi/8$ and Hadamard gates based on Berry and Aharonov-Anandan phases. For two interacting qubits in a rotating field, we find that it is always impossible to construct a two-qubit gate based on Berry phases, or based on Aharonov-Anandan phases when the gyromagnetic ratios of the two qubits are equal. In implementing a universal set of quantum gates, one may combine geometric $pi/8$ and Hadamard gates and dynamical $sqrt{rm SWAP}$ gate.