We show here a series of energy gaps as in Hofstadters butterfly, which have been shown to exist by Koshino et al [Phys. Rev. Lett. 86, 1062 (2001)] for anisotropic three-dimensional (3D) periodic systems in magnetic fields $Vec{B}$, also arise in the isotropic case unless $Vec{B}$ points in high-symmetry directions. Accompanying integer quantum Hall conductivities $(sigma_{xy}, sigma_{yz}, sigma_{zx})$ can, surprisingly, take values $propto (1,0,0), (0,1,0), (0,0,1)$ even for a fixed direction of $Vec{B}$ unlike in the anisotropic case. We can intuitively explain the high-magnetic field spectra and the 3D QHE in terms of quantum mechanical hopping by introducing a ``duality, which connects the 3D system in a strong $Vec{B}$ with another problem in a weak magnetic field $(propto 1/B)$.