Using cosmological hydrodynamic simulations with physically motivated models of super-massive black hole (SMBH) formation and growth, we compare the assembly of Milky Way-mass ( $M_{mathrm{vir}} approx 7 times 10^{11}$ $M_{odot}$ at $z = 0$) galaxies in cold dark matter (CDM) and self-interacting dark matter (SIDM) models. Our SIDM model adopts a constant cross-section of 1 cm$^2$/g. We find that SMBH formation is suppressed in the early universe due to SIDM coring. SMBH-SMBH mergers are also suppressed in SIDM as a consequence of the lower number of SMBHs formed. Lack of initial merger-driven SMBH growth in turn delays SMBH growth by billions of years in SIDM compared to CDM. Further, we find that this delayed growth suppresses SMBH accretion in the largest progenitors of the main SIDM galaxies during the first 5 Gyrs of their evolution. Nonetheless, by $z = 0.8$ the CDM and SIDM SMBH masses differ only by around 0.2 dex, so that both remain compatible with the $M_{BH}-M_{*}$ relation. We show that the reduced accretion causes the SIDM SMBHs to less aggressively regulate star formation in their host galaxies than their CDM counterparts, resulting in a factor of 3 or more stars being produced over the lifetime of the SIDM galaxies compared to the CDM galaxies. Our results highlight a new way in which SIDM can affect the growth and merger history of SMBHs and ultimately give rise to very different galaxy evolution compared to the classic CDM model.