Physical platforms such as trapped ions suffer from coherent noise where errors manifest as rotations about a particular axis and can accumulate over time. We investigate passive mitigation through decoherence free subspaces, requiring the noise to preserve the code space of a stabilizer code, and to act as the logical identity operator on the protected information. Thus, we develop necessary and sufficient conditions for all transversal $Z$-rotations to preserve the code space of a stabilizer code, which require the weight-$2$ $Z$-stabilizers to cover all the qubits that are in the support of some $X$-component. Further, the weight-$2$ $Z$-stabilizers generate a direct product of single-parity-check codes with even block length. By adjusting the size of these components, we are able to construct a large family of QECC codes, oblivious to coherent noise, that includes the $[[4L^2, 1, 2L]]$ Shor codes. Moreover, given $M$ even and any $[[n,k,d]]$ stabilizer code, we can construct an $[[Mn, k, ge d]]$ stabilizer code that is oblivious to coherent noise. If we require that transversal $Z$-rotations preserve the code space only up to some finite level $l$ in the Clifford hierarchy, then we can construct higher level gates necessary for universal quantum computation. The $Z$-stabilizers supported on each non-zero $X$-component form a classical binary code C, which is required to contain a self-dual code, and the classical Gleasons theorem constrains its weight enumerator. The conditions for a stabilizer code being preserved by transversal $2pi/2^l$ $Z$-rotations at $4 le l le l_{max} <infty$ level in the Clifford hierarchy lead to generalizations of Gleasons theorem that may be of independent interest to classical coding theorists.