When, and how, did the first galaxies and supermassive black holes (SMBH) form, and how did they reionization the Universe? First galaxy formation and cosmic reionization are among the last frontiers in studies of cosmic structure formation. We delineate the detailed astrophysical probes of early galaxy and SMBH formation afforded by observations at centimeter through submillimeter wavelengths. These observations include studies of the molecular gas (= the fuel for star formation in galaxies), atomic fine structure lines (= the dominant ISM gas coolant), thermal dust continuum emission (= an ideal star formation rate estimator), and radio continuum emission from star formation and relativistic jets. High resolution spectroscopic imaging can be used to study galaxy dynamics and star formation on sub-kpc scales. These cm and mm observations are the necessary compliment to near-IR observations, which probe the stars and ionized gas, and X-ray observations, which reveal the AGN. Together, a suite of revolutionary observatories planned for the next decade from centimeter to X-ray wavelengths will provide the requisite panchromatic view of the complex processes involved in the formation of the first generation of galaxies and SMBHs, and cosmic reionization.