We present limits on sterile neutrino mixing using 4,438 live-days of atmospheric neutrino data from the Super-Kamiokande experiment. We search for fast oscillations driven by an eV$^2$-scale mass splitting and for oscillations into sterile neutrinos instead of tau neutrinos at the atmospheric mass splitting. When performing both these searches we assume that the sterile mass splitting is large, allowing $sin^2(Delta m^2 L/4E)$ to be approximated as $0.5$, and we assume that there is no mixing between electron neutrinos and sterile neutrinos ($|U_{e4}|^2 = 0$). No evidence of sterile oscillations is seen and we limit $|U_{mu4}|^2$ to less than 0.041 and $|U_{tau4}|^2$ to less than 0.18 for $Delta m^2 > 0.8$ eV$^2$ at the 90% C.L. in a 3+1 framework. The approximations that can be made with atmospheric neutrinos allow these limits to be easily applied to 3+N models, and we provide our results in a generic format to allow comparisons with other sterile neutrino models.