Exact synthesis is a tool used in algorithms for approximating an arbitrary qubit unitary with a sequence of quantum gates from some finite set. These approximation algorithms find asymptotically optimal approximations in probabilistic polynomial time, in some cases even finding the optimal solution in probabilistic polynomial time given access to an oracle for factoring integers. In this paper, we present a common mathematical structure underlying all results related to the exact synthesis of qubit unitaries known to date, including Clifford+T, Clifford-cyclotomic and V-basis gate sets, as well as gates sets induced by the braiding of Fibonacci anyons in topological quantum computing. The framework presented here also provides a means to answer questions related to the exact synthesis of unitaries for wide classes of other gate sets, such as Clifford+T+V and SU(2) level k anyons.