We investigate the stellar populations of Lyman alpha emitters (LAEs) at z=5.7 and 6.6 in a 0.65 deg^2 sky of the Subaru/XMM-Newton Deep Survey (SXDS) Field, using deep images taken with Subaru/Suprime-Cam, UKIRT/WFCAM, and Spitzer/IRAC. We produce stacked multiband images at each redshift from 165 (z=5.7) and 91 (z=6.6) IRAC-undetected objects, to derive typical spectral energy distributions (SEDs) of z~6-7 LAEs for the first time. The stacked LAEs have as blue UV continua as the HST/WFC3 z-dropout galaxies of similar Muv, with a spectral slope beta ~ -3, but at the same time they have red UV-to-optical colors with detection in the 3.6um band. Using SED fitting we find that the stacked LAEs have low stellar masses of ~(3-10)*10^7 Msun, very young ages of ~1-3 Myr, negligible dust extinction, and strong nebular emission from the ionized interstellar medium, although the z=6.6 object is fitted similarly well with high-mass models without nebular emission; inclusion of nebular emission reproduces the red UV-to-optical color while keeping the UV color sufficiently blue. We infer that typical LAEs at z~6-7 are building blocks of galaxies seen at lower redshifts. We find a tentative decrease in the Lyman alpha escape fraction from z=5.7 to 6.6, which may imply an increase in the intergalactic medium neutral fraction. From the minimum contribution of nebular emission required to fit the observed SEDs, we place an upper limit on the escape fraction of ionizing photons to be f_esc^ion~0.6 at z=5.7 and ~0.9 at z=6.6. We also compare the stellar populations of our LAEs with that of stacked HST/WFC3 z-dropout galaxies.