We construct a map between a class of codes over $F_4$ and a family of non-rational Narain CFTs. This construction is complementary to a recently introduced relation between quantum stabilizer codes and a class of rational Narain theories. From the modular bootstrap point of view we formulate a polynomial ansatz for the partition function which reduces modular invariance to a handful of algebraic easy-to-solve constraints. For certain small values of central charge our construction yields optimal theories, i.e. those with the largest value of the spectral gap.