Globular clusters (GCs) formed when the Milky Way experienced a phase of rapid assembly. We use the wealth of information contained in the Galactic GC population to quantify the properties of the satellite galaxies from which the Milky Way assembled. To achieve this, we train an artificial neural network on the E-MOSAICS cosmological simulations of the co-formation and co-evolution of GCs and their host galaxies. The network uses the ages, metallicities, and orbital properties of GCs that formed in the same progenitor galaxies to predict the stellar masses and accretion redshifts of these progenitors. We apply the network to Galactic GCs associated with five progenitors: {it Gaia}-Enceladus, the Helmi streams, Sequoia, Sagittarius, and the recently discovered, `low-energy GCs, which provide an excellent match to the predicted properties of the enigmatic galaxy `Kraken. The five galaxies cover a narrow stellar mass range [$M_star=(0.6{-}4.6)times10^8~{rm M}_odot$], but have widely different accretion redshifts ($z_{rm acc}=0.57{-}2.65$). All accretion events represent minor mergers, but Kraken likely represents the most major merger ever experienced by the Milky Way, with stellar and virial mass ratios of $r_{M_star}=1$:$31^{+34}_{-16}$ and $r_{M_{rm h}}=1$:$7^{+4}_{-2}$, respectively. The progenitors match the $z=0$ relation between GC number and halo virial mass, but have elevated specific frequencies, suggesting an evolution with redshift. Even though these progenitors likely were the Milky Ways most massive accretion events, they contributed a total mass of only $log{(M_{rm star,tot}/{rm M}_odot)}=9.0pm0.1$, similar to the stellar halo. This implies that the Milky Way grew its stellar mass mostly by in-situ star formation. We conclude by organising these accretion events into the most detailed reconstruction to date of the Milky Ways merger tree.