As a basic requirement of the renormalization group invariance, any physical observable must be independent of the choice of both the renormalization scheme and the initial renormalization scale. In this paper, we show that by using the newly suggested $C$-scheme coupling, one can obtain a demonstration that the {it Principle of Maximum Conformality} prediction is scheme-independent to all-orders for any renormalization schemes, thus satisfying all of the conditions of the renormalization group invariance. We illustrate these features for the non-singlet Adler function and for $tau $ decay to $ u +$ hadrons at the four-loop level.