We comment on the recently introduced Gauss-Bonnet gravity in four dimensions. We argue that it does not make sense to consider this theory to be defined by a set of $Dto 4$ solutions of the higher-dimensional Gauss-Bonnet gravity. We show that a well-defined $Dto 4$ limit of Gauss-Bonnet Gravity is obtained generalizing a method employed by Mann and Ross to obtain a limit of the Einstein gravity in $D=2$ dimensions. This is a scalar-tensor theory of the Horndeski type obtained by a dimensional reduction methods. By considering simple spacetimes beyond spherical symmetry (Taub-NUT spaces) we show that the naive limit of the higher-dimensional theory to four dimensions is not well defined and contrast the resultant metrics with the actual solutions of the new theory.