We show that the critical behaviour of two- and three-dimensional frustrated magnets cannot reliably be described from the known five- and six-loops perturbative renormalization group results. Our conclusions are based on a careful re-analysis of the resummed perturbative series obtained within the zero momentum massive scheme. In three dimensions, the critical exponents for XY and Heisenberg spins display strong dependences on the parameters of the resummation procedure and on the loop order. This behaviour strongly suggests that the fixed points found are in fact spurious. In two dimensions, we find, as in the O(N) case, that there is apparent convergence of the critical exponents but towards erroneous values. As a consequence, the interesting question of the description of the crossover/transition induced by Z2 topological defects in two-dimensional frustrated Heisenberg spins remains open.