The goal of the KArlsruhe TRItrium Neutrino (KATRIN) experiment is the determination of the effective electron antineutrino mass with a sensitivity of 0.2 eV/c$^2$ at 90% C.L. This goal can only be achieved with a very low background level in the order of 0.01 counts per second. A possible background source is $alpha$-decays on the inner surface of the KATRIN Main Spectrometer. Two $alpha$-sources, $^{223}$Ra and $^{228}$Th, were installed at the KATRIN Main Spectrometer with the purpose of temporarily increasing the background in order to study $alpha$-decay induced background processes. In this paper, we present a possible background generation mechanism and measurements performed with these two radioactive sources. Our results show a clear correlation between $alpha$-activity on the inner spectrometer surface and background from the volume of the spectrometer. Two key characteristics of the Main Spectrometer background -the dependency on the inner electrode offset potential, and the radial distribution - could be reproduced with this artificially induced background. These findings indicate a high contribution of $alpha$-decay induced events to the residual KATRIN background.