We present an explicit matrix algebra regularization of the algebra of volume-preserving diffeomorphisms of the $n$-torus. That is, we approximate the corresponding classical Nambu brackets using $mathfrak{sl}(N^{lceiltfrac{n}{2}rceil},mathbb{C})$-matrices equipped with the finite bracket given by the completely anti-symmetrized matrix product, such that the classical brackets are retrieved in the $Nrightarrow infty$ limit. We then apply this approximation to the super $4$-brane in $9$ dimensions and give a regularized action in analogy with the matrix regularization of the supermembrane. This action exhibits a reduced gauge symmetry that we discuss from the viewpoint of $L_infty$-algebras in a slight generalization to the construction of Lie $2$-algebras from Bagger-Lambert $3$-algebras.