The lowest luminosity (L < 10^5 L_sun) Milky Way satellite galaxies represent the extreme lower limit of the galaxy luminosity function. These ultra-faint dwarfs are the oldest, most dark matter-dominated, most metal-poor, and least chemically evolved stellar systems known. They therefore provide unique windows into the formation of the first galaxies and the behavior of dark matter on small scales. In this review, we summarize the discovery of ultra-faint dwarfs in the Sloan Digital Sky Survey in 2005, and the subsequent observational and theoretical progress in understanding their nature and origin. We describe their stellar kinematics, chemical abundance patterns, structural properties, stellar populations, orbits, and luminosity function, and what can be learned from each type of measurement. We conclude that: (1) in most cases, the stellar velocity dispersions of ultra-faint dwarfs are robust against systematic uncertainties such as binary stars and foreground contamination; (2) the chemical abundance patterns of stars in ultra-faint dwarfs require two sources of r-process elements, one of which can likely be attributed to neutron star mergers; (3) even under conservative assumptions, only a small fraction of ultra-faint dwarfs may have suffered significant tidal stripping of their stellar components; (4) determining the properties of the faintest dwarfs out to the virial radius of the Milky Way will require very large investments of observing time with future telescopes. Finally, we offer a look forward at the observations that will be possible with future facilities as the push toward a complete census of the Local Group dwarf galaxy population continues.