In the presence of appropriate non-minimal couplings between a scalar field and the curvature squared Gauss-Bonnet (GB) term, compact objects such as neutron stars and black holes (BHs) can spontaneously scalarize, becoming a preferred vacuum. Such strong gravity phase transitions have attracted considerable attention recently. The non-minimal coupling functions that allow this mechanism are, however, always postulated ad hoc. Here we point out that families of such functions naturally emerge in the context of Higgs--Chern-Simons gravity models, which are found as dimensionally descents of higher dimensional, purely topological, Chern-Pontryagin non-Abelian densities. As a proof of concept, we study spherically symmetric scalarized BH solutions in a particular Einstein-GB-scalar field model, whose coupling is obtained from this construction, pointing out novel features and caveats thereof. The possibility of vectorization is also discussed, since this construction also originates vector fields non-minimally coupled to the GB invariant.