Vector theories with non-linear derivative self-interactions that break gauge symmetries have been shown to have interesting cosmological applications. In this paper we introduce a way to spontaneously break the gauge symmetry and construct these theories via a Higgs mechanism. In addition to the purely gauge field interactions, our method generates new ghost-free scalar-vector interactions between the Higgs field and the gauge boson. We show how these additional terms are found to reduce, in a suitable decoupling limit, to scalar bi-Galileon interactions between the Higgs field and Goldstone bosons. Our formalism is first developed in the context of abelian symmetry, which allows us to connect with earlier work on the extension of the Proca action. We then show how this formalism is straightforwardly generalised to generate theories with non-abelian symmetry.