Axisymmetric disks of eccentric Kepler orbits are vulnerable to an instability which causes orbits to exponentially grow in inclination, decrease in eccentricity, and cluster in their angle of pericenter. Geometrically, the disk expands to a cone shape which is asymmetric about the mid-plane. In this paper, we describe how secular gravitational torques between individual orbits drive this inclination instability. We derive growth timescales for a simple two-orbit model using a Gauss $N$-ring code, and generalize our result to larger $N$ systems with $N$-body simulations. We find that two-body relaxation slows the growth of the instability at low $N$ and that angular phase coverage of orbits in the disk is important at higher $N$. As $N to infty$, the e-folding timescale converges to that expected from secular theory.