We consider the steepest rate at which the power spectrum from single field inflation can grow, with the aim of providing a simple explanation for the $k^4$ growth found recently. With this explanation in hand we show that a slightly steeper $k^5 (log k )^2$ growth is in fact possible. Moreover, we argue that the power spectrum after a steep growth cannot immediately decay, but must remain large for the $k$ modes which exit during a $sim2$ e-fold period. We also briefly consider how a strong growth can affect the spectral index of longer wavelengths preceding the growth, and show that even the conversion of isocurvature modes likely cannot lead to a stronger growth. These results have implications for the formation of primordial black holes, and other phenomena which require a large amplitude of power spectrum at short scales.