We consider the strong dynamics associated with a composite Higgs model that simultaneously produces dynamical axions and solves the strong CP problem. The strong dynamics arises from a new $Sp$ or $SU(4)$ hypercolor gauge group containing QCD colored hyperfermions that confines at a high scale. The hypercolor global symmetry is weakly gauged by the Standard Model electroweak gauge group and an enlarged color group, $SU(N+3) times SU(N)$. When hyperfermion condensates form, they not only lead to an $SU(5)/SO(5)$ composite Higgs model but also spontaneously break the enlarged color group to $SU(3)_c times SU(N)_D$. At lower energies, the $SU(N)_D$ group confines, producing two dynamical axions that eliminates all CP violation. Furthermore, small instantons from the $SU(N)$ group can enhance the axion mass, giving rise to TeV scale axion masses that can be detected at collider experiments. Our model provides a way to unify the composite Higgs with dynamical axions, without introducing new elementary scalar fields, while also extending the range of axion masses that addresses the strong CP problem.