The topological semimetal $beta$-Ag2Se features a Kramers Weyl node at the origin in momentum space and a quadruplet of spinless Weyl nodes, which are annihilated by spin-orbit coupling. We show that single crystalline $beta$-Ag2Se manifests giant Shubnikov-de Haas oscillations in the longitudinal magnetoresistance which stem from a small electron pocket that can be driven beyond the quantum limit by a field less than 9 T. This small electron pocket is a remainder of the spin-orbit annihilatedWeyl nodes and thus encloses a Berry-phase structure. Moreover, we observed a negative longitudinal magnetoresistance when the magnetic field is beyond the quantum limit. Our experimental findings are complemented by thorough theoretical band structure analyses of this Kramers Weyl semimetal candidate, including first-principle calculations and an effective k*p model.