In previous work of the first author and Jibladze, the $E_3$-term of the Adams spectral sequence was described as a secondary derived functor, defined via secondary chain complexes in a groupoid-enriched category. This led to computations of the $E_3$-term using the algebra of secondary cohomology operations. In work with Blanc, an analogous description was provided for all higher terms $E_m$. In this paper, we introduce $2$-track algebras and tertiary chain complexes, and we show that the $E_4$-term of the Adams spectral sequence is a tertiary Ext group in this sense. This extends the work with Jibladze, while specializing the work with Blanc in a way that should be more amenable to computations.