Flat electronic bands, characteristic of magic-angle twisted bilayer graphene (TBG), host a wealth of correlated phenomena. Early theoretical considerations suggested that, at the magic angle, the Dirac velocity vanishes and the entire width of the moire bands becomes extremely narrow. Yet, this scenario contradicts experimental studies that reveal a finite Dirac velocity as well as bandwidths significantly larger than predicted. Here we use spatially resolved spectroscopy in finite and zero magnetic fields to examine the electronic structure of moire bands and their intricate connection to correlated phases. By following the relative shifts of Landau levels in finite fields, we detect filling-dependent band flattening, that unexpectedly starts already at ~1.3 degrees, well above the magic angle and hence nominally in the weakly correlated regime. We further show that, as the twist angle is reduced, the moire bands become maximally flat at progressively lower doping levels. Surprisingly, when the twist angles reach values for which the maximal flattening occurs at approximate filling of $-2$, $+1$,$+2$,$+3$ electrons per moire unit cell, the corresponding zero-field correlated phases start to emerge. Our observations are corroborated by calculations that incorporate an interplay between the Coulomb charging energy and exchange interactions; together these effects produce band flattening and hence a significant density-of-states enhancement that facilitates the observed symmetry-breaking cascade transitions. Besides emerging phases pinned to integer fillings, we also experimentally identify a series of pronounced correlation-driven band deformations and soft gaps in a wider doping range around $pm 2$ filling where superconductivity is expected. Our results highlight the role of interaction-driven band-flattening in forming robust correlated phases in TBG.