Future high spectroscopic resolution galaxy surveys will observe galaxies with nearly full-sky footprints. Modeling the galaxy clustering for these surveys, therefore, must include the wide-angle effect with narrow redshift binning. In particular, when the redshift-bin size is comparable to the typical peculiar velocity field, the nonlinear redshift-space distortion (RSD) effect becomes important. A naive projection of the Fourier-space RSD model to spherical harmonic space leads to diverging expressions. In this paper we present a general formalism of projecting the higher-order RSD terms into spherical harmonic space. We show that the nonlinear RSD effect, including the fingers-of-God (FoG), can be entirely attributed to a modification of the radial window function. We find that while linear RSD enhances the harmonic-space power spectrum, unlike the three-dimensional case, the enhancement decreases on small angular-scales. The fingers-of-God suppress the angular power spectrum on all transverse scales if the bin size is smaller than $Delta r lesssim pi sigma_u$; for example, the radial bin sizes corresponding to a spectral resolution $R=lambda/Delta lambda$ of a few hundred satisfy the condition. We also provide the flat-sky approximation which reproduces the full calculation to sub-percent accuracy.