In order to construct a gauge invariant two-point function in a Yang-Mills theory, we propose the use of the all-order gauge invariant transverse configurations A^h. Such configurations can be obtained through the minimization of the functional A^2_{min} along the gauge orbit within the BRST invariant formulation of the Gribov-Zwanziger framework recently put forward in [1,2] for the class of the linear covariant gauges. This correlator turns out to provide a characterization of non-perturbative aspects of the theory in a BRST invariant and gauge parameter independent way. In particular, it turns out that the poles of <A^h A^h> are the same as those of the transverse part of the gluon propagator, which are also formally shown to be independent of the gauge parameter entering the gauge condition through the Nielsen identities. The latter follow from the new exact BRST invariant formulation introduced before. Moreover, the correlator <A^h A^h> enables us to attach a BRST invariant meaning to the possible positivity violation of the corresponding temporal Schwinger correlator, giving thus for the first time a consistent, gauge parameter independent, setup to adopt the positivity violation of <A^h A^h> as a signature for gluon confinement. Finally, in the context of gauge theories supplemented with a fundamental Higgs field, we use <A^h A^h> to probe the pole structure of the massive gauge boson in a gauge invariant fashion.