We use data taken as part of HST/WFC3 observations of the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS) to identify massive and evolved galaxies at 3<z<4.5. This is performed using the strength of the Balmer break feature at rest-frame 3648A, which is a diagnostic of the age of the stellar population in galaxies. Using WFC3 H-band selected catalog for the CANDELS GOODS-S field and deep multi-waveband photometry from optical (HST) to mid-infrared (Spitzer) wavelengths, we identify a population of old and evolved post-starburst galaxies based on the strength of their Balmer breaks (Balmer Break Galaxies- BBGs). The galaxies are also selected to be bright in rest-frame near-IR wavelengths and hence, massive. We identify a total of 16 BBGs. Fitting the spectral energy distribution (SED) of the BBGs show that the candidate galaxies have average estimated ages of ~800 Myr and average stellar masses of ~5x10^10 M_sun, consistent with being old and massive systems. Two of our BBG candidates are also identified by the criteria that is sensitive to star forming galaxies (LBG selection). We find a number density of ~3.2x10^-5 Mpc^-3 for the BBGs corresponding to a mass density of ~2.0x10^6 M_sun/Mpc^3 in the redshift range covering the survey. Given the old age and the passive evolution, it is argued that some of these objects formed the bulk of their mass only a few hundred million years after the Big Bang.