We present thermopower measurements on Yb(Rh$_{1-x}$Co$_x$)$_2$Si$_2$. Upon Co substitution the Kondo temperature is decreasing and the single large thermopower minimum observed for YbRh$_2$Si$_2$ splits into two minima. Simultaneously, the absolute thermopower values are strongly reduced due to a weaker exchange coupling between the $4f$ and the conduction electron states with increasing $x$. Pure YbCo$_2$Si$_2$ is considered a stable, trivalent system. Nevertheless, we still observe two minima in the thermopower indicative of weak residual Kondo scattering. This is in line with results from photo emission spectroscopy revealing a tiny contribution from Yb$^{2+}$. The value at the high-$T$ minimum in $S(T)$ is found to be proportional to the Sommerfeld coefficient for the whole series. This unexpected finding is discussed in relation to recent measurements of the valence and Fermi surface evolution with temperature.