Following work of Colding-Minicozzi, we define a notion of entropy for connections over $mathbb R^n$ which has shrinking Yang-Mills solitons as critical points. As in Colding-Minicozzi, this entropy is defined implicitly, making it difficult to work with analytically. We prove a theorem characterizing entropy stability in terms of the spectrum of a certain linear operator associated to the soliton. This leads furthermore to a gap theorem for solitons. These results point to a broader strategy of studying generic singularities of Yang-Mills flow, and we discuss the differences in this strategy in dimension $n=4$ versus $n geq 5$.