We show that, regardless of the dimension of the Hilbert space, there exists no set of rays revealing state-independent contextuality with less than 13 rays. This implies that the set proposed by Yu and Oh in dimension three [Phys. Rev. Lett. 108, 030402 (2012)] is actually the minimal set in quantum theory. This contrasts with the case of Kochen-Specker sets, where the smallest set occurs in dimension four.