We develop a theory for the magnon Kerr effect in a cavity magnonics system, consisting of magnons in a small yttrium iron garnet (YIG) sphere strongly coupled to cavity photons, and use it to study the bistability in this hybrid system. To have a complete picture of the bistability phenomenon, we analyze two different cases in driving the cavity magnonics system, i.e., directly pumping the YIG sphere and the cavity, respectively. In both cases, the magnon frequency shifts due to the Kerr effect exhibit a similar bistable behavior but the corresponding critical powers are different. Moreover, we show how the bistability of the system can be demonstrated using the transmission spectrum of the cavity. Our results are valid in a wide parameter regime and generalize the theory of bistability in a cavity magnonics system.