We show that the restricted Lie algebra structure on Hochschild cohomology is invariant under stable equivalences of Morita type between self-injective algebras. Thereby we obtain a number of positive characteristic stable invariants, such as the $p$-toral rank of $mathrm{HH}^1(A,A)$. We also prove a more general result concerning Iwanaga-Gorenstein algebras, using a more general notion of stable equivalences of Morita type. Several applications are given to commutative algebra and modular representation theory. These results are proven by first establishing the stable invariance of the $B_infty$-structure of the Hochschild cochain complex. In the appendix we explain how the $p$-power operation on Hochschild cohomology can be seen as an artifact of this $B_infty$-structure. In particular, we establish well-definedness of the $p$-power operation, following some -- originally topological -- methods due to May, Cohen and Turchin, using the language of operads.