In this paper, we provide a thorough study on the expansion of single trace Einstein-Yang-Mills amplitudes into linear combination of color-ordered Yang-Mills amplitudes, from various different perspectives. Using the gauge invariance principle, we propose a recursive construction, where EYM amplitude with any number of gravitons could be expanded into EYM amplitudes with less number of gravitons. Through this construction, we can write down the complete expansion of EYM amplitude in the basis of color-ordered Yang-Mills amplitudes. As a byproduct, we are able to write down the polynomial form of BCJ numerator, i.e., numerators satisfying the color-kinematic duality, for Yang-Mills amplitude. After the discussion of gauge invariance, we move to the BCFW on-shell recursion relation and discuss how the expansion can be understood from the on-shell picture. Finally, we show how to interpret the expansion from the aspect of KLT relation and the way of evaluating the expansion coefficients efficiently.