Statistical observations of the Epoch of Reionization using the 21 cm line of neutral hydrogen have the potential to revolutionize our understanding of structure formation and the first luminous objects. However, these observations are complicated by a host of strong foreground sources. Several foreground removal techniques have been proposed in the literature, and it has been assumed that these would be used in combination to reveal the Epoch of Reionization (EOR) signal. By studying the characteristic subtraction errors of the proposed foreground removal techniques, we identify an additional subtraction stage that can further reduce the EOR foreground contamination, and study the interactions between the foreground removal algorithms. This enables us to outline a comprehensive foreground removal strategy that incorporates all previously proposed subtraction techniques. Using this foreground removal framework and the characteristic subtraction errors, we discuss the complementarity of different foreground removal techniques and the implications for array design and the analysis of EOR data.