We search for galaxies with a strong Balmer break (Balmer Break Galaxies; BBGs) at $z sim 6$ over a 0.41 deg$^2$ effective area in the COSMOS field. Based on rich imaging data, including data obtained with the Atacama Large Millimeter/submillimeter Array (ALMA), three candidates are identified by their extremely red $K - [3.6]$ colors as well as by non-detection in X-ray, optical, far-infrared (FIR), and radio bands. The non-detection in the deep ALMA observations suggests that they are not dusty galaxies but BBGs at $z sim 6$, although contamination from Active Galactic Nuclei (AGNs) at $z sim 0$ cannot be completely ruled out for the moment. Our spectral energy distribution (SED) analyses reveal that the BBG candidates at $z sim 6$ have stellar masses of $approx 5 times 10^{10} M_{odot}$ dominated by old stellar populations with ages of $gtrsim 700$ Myr. Assuming that all the three candidates are real BBGs at $z sim 6$, we estimate the stellar mass density (SMD) to be $2.4^{+2.3}_{-1.3} times 10^{4} M_{odot}$ Mpc$^{-3}$. This is consistent with an extrapolation from the lower redshift measurements. The onset of star formation in the three BBG candidates is expected to be several hundred million years before the observed epoch of $z sim 6$. We estimate the star-formation rate density (SFRD) contributed by progenitors of the BBGs to be 2.4 -- 12 $times 10^{-5} M_{odot}$ yr$^{-1} $Mpc$^{-3}$ at $z > 14$ (99.7% confidence range). Our result suggests a smooth evolution of the SFRD beyond $z = 8$.