The Energetic X-ray Imaging Survey Telescope (EXIST) mission concept is optimized for study of high-z GRBs as probes of the early Universe. With a High Energy Telescope (HET) incorporating a 4.5m^2 5-600keV (CZT; 0.6mm pixels) detector plane for coded aperture imaging a 90deg x 70deg (>10% coding fraction) field of view with 2 resolution and <20 (90% conf.) positions for >5 sigma sources, EXIST will perform rapid (<200sec) slews onto GRBs. Prompt images and spectra are obtained with a co-aligned soft X-ray telescope (SXI; 0.1 - 10keV) and with a 1.1m optical-IR telescope (IRT) simultaneously in 4 bands (0.3 - 0.52micron, 0.52 - 0.9micron, 0.9 - 1.38micron, and 1.38 - 2.3micron). An initial image (100s) will yield prompt identification within the HET error circle from a <2 prompt SXI position; or from VIS vs. IR dropouts or variability. An autonomous spacecraft re-point (<30) will then put the GRB on a 0.3 x 4 slit for either R = 3000 (for AB <21) or R =30 (for AB ~21-25) prompt spectra over the 0.3 - 0.9 micron and 0.9 - 2.3 micron bands. This will provide onboard redshifts within ~500-2000sec for most GRBs, reaching z ~20 (for Lyman-alpha breaks) if such GRBs exist, and spectra for studies of the host galaxy and local re-ionization patchiness as well as intervening cosmic structure. With ~600 GRBs/yr expected, including ~7-10% expected at z >7, EXIST will open a new era in studies of the early Universe as well as carry out a rich program of AGN and transient-source science. An overview of the GRB science objectives and a brief discussion of the overall mission design and operations is given, and example high-z GRB IRT spectra are shown. EXIST is being proposed to the Astro2010 Decadal Survey as a 5 year Medium Class mission that could be launched as early as 2017.