This article addresses the regularity issue for stationary or minimizing fractional harmonic maps into spheres of order $sin(0,1)$ in arbitrary dimensions. It is shown that such fractional harmonic maps are $C^infty$ away from a small closed singular set. The Hausdorff dimension of the singular set is also estimated in terms of $sin(0,1)$ and the stationarity/minimality assumption.