The discrete prolate spheroidal sequences (DPSSs) are a set of orthonormal sequences in $ell_2(mathbb{Z})$ which are strictly bandlimited to a frequency band $[-W,W]$ and maximally concentrated in a time interval ${0,ldots,N-1}$. The timelimited DPSSs (sometimes referred to as the Slepian basis) are an orthonormal set of vectors in $mathbb{C}^N$ whose discrete time Fourier transform (DTFT) is maximally concentrated in a frequency band $[-W,W]$. Due to these properties, DPSSs have a wide variety of signal processing applications. The DPSSs are the eigensequences of a timelimit-then-bandlimit operator and the Slepian basis vectors are the eigenvectors of the so-called prolate matrix. The eigenvalues in both cases are the same, and they exhibit a particular clustering behavior -- slightly fewer than $2NW$ eigenvalues are very close to $1$, slightly fewer than $N-2NW$ eigenvalues are very close to $0$, and very few eigenvalues are not near $1$ or $0$. This eigenvalue behavior is critical in many of the applications in which DPSSs are used. There are many asymptotic characterizations of the number of eigenvalues not near $0$ or $1$. In contrast, there are very few non-asymptotic results, and these dont fully characterize the clustering behavior of the DPSS eigenvalues. In this work, we establish two novel non-asymptotic bounds on the number of DPSS eigenvalues between $epsilon$ and $1-epsilon$. Also, we obtain bounds detailing how close the first $approx 2NW$ eigenvalues are to $1$ and how close the last $approx N-2NW$ eigenvalues are to $0$. Furthermore, we extend these results to the eigenvalues of the prolate spheroidal wave functions (PSWFs), which are the continuous-time version of the DPSSs. Finally, we present numerical experiments demonstrating the quality of our non-asymptotic bounds on the number of DPSS eigenvalues between $epsilon$ and $1-epsilon$.