The Generalized Lax Conjecture asks whether every hyperbolicity cone is a section of a semidefinite cone of sufficiently high dimension. We prove that the space of hyperbolicity cones of hyperbolic polynomials of degree $d$ in $n$ variables contains $(n/d)^{Omega(d)}$ pairwise distant cones in a certain metric, and therefore that any semidefinite representation of such cones must have dimension at least $(n/d)^{Omega(d)}$ (even if a small approximation is allowed). The proof contains several ingredients of independent interest, including the identification of a large subspace in which the elementary symmetric polynomials lie in the relative interior of the set of hyperbolic polynomials, and quantitati