We perform the first spatially-resolved stellar population study of galaxies in the early universe (z = 3.5 - 6.5), utilizing the Hubble Space Telescope Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS) imaging dataset over the GOODS-S field. We select a sample of 418 bright and extended galaxies at z = 3.5 - 6.5 from a parent sample of ~ 8000 photometric-redshift selected galaxies from Finkelstein et al. (2015). We first examine galaxies at 3.5< z < 4.0 using additional deep K-band survey data from the HAWK-I UDS and GOODS Survey (HUGS) which covers the 4000A break at these redshifts. We measure the stellar mass, star formation rate, and dust extinction for galaxy inner and outer regions via spatially-resolved spectral energy distribution fitting based on a Markov Chain Monte Carlo algorithm. By comparing specific star formation rates (sSFRs) between inner and outer parts of the galaxies we find that the majority of galaxies with the high central mass densities show evidence for a preferentially lower sSFR in their centers than in their outer regions, indicative of reduced sSFRs in their central regions. We also study galaxies at z ~ 5 and 6 (here limited to high spatial resolution in the rest-frame ultraviolet only), finding that they show sSFRs which are generally independent of radial distance from the center of the galaxies. This indicates that stars are formed uniformly at all radii in massive galaxies at z ~ 5 - 6, contrary to massive galaxies at z < 4.