In an SU(N) gauge field theory, the n-point Green functions, namely, propagators and vertices, transform under the simultaneous local gauge variations of the gluon vector potential and the quark matter field in such a manner that the physical observables remain invariant. In this article, we derive this intrinsically non perturbative transformation law for the quark propagator within the system of covariant gauges. We carry out its explicit perturbative expansion till O(g_s^6) and, for some terms, till O(g_s^8). We study the implications of this transformation for the quark-anti-quark condensate, multiplicative renormalizability of the massless quark propagator, as well as its relation with the quark-gluon vertex at the one-loop order. Setting the color factors C_F=1 and C_A=0, Landau-Khalatnikov-Fradkin transformation for the abelian case of quantum electrodynamics is trivially recovered.