We present an analysis of all prime HST legacy fields spanning >800 arcmin^2 for the search of z~10 galaxy candidates and the study of their UV luminosity function (LF). In particular, we present new z~10 candidates selected from the full Hubble Frontier Field (HFF) dataset. Despite the addition of these new fields, we find a low abundance of z~10 candidates with only 9 reliable sources identified in all prime HST datasets that include the HUDF09/12, the HUDF/XDF, all the CANDELS fields, and now the HFF survey. Based on this comprehensive search, we find that the UV luminosity function decreases by one order of magnitude from z~8 to z~10 at all luminosities over a four magnitude range. This also implies a decrease of the cosmic star-formation rate density by an order of magnitude within 170 Myr from z~8 to z~10. We show that this accelerated evolution compared to lower redshift can entirely be explained by the fast build-up of the dark matter halo mass function at z>8. Consequently, the predicted UV LFs from several models of galaxy formation are in good agreement with this observed trend, even though the measured UV LF lies at the low end of model predictions. In particular, the number of only 9 observed candidate galaxies is lower, by ~50%, than predicted by galaxy evolution models. The difference is generally still consistent within the Poisson and cosmic variance uncertainties. However, essentially all models predict larger numbers than observed. We discuss the implications of these results in light of the upcoming James Webb Space Telescope mission, which is poised to find much larger samples of z~10 galaxies as well as their progenitors at less than 400 Myr after the Big Bang.