The finding of bulk quantum oscillations in the bulk Kondo insulator SmB$_6$, which has been proposed to be a correlated topological insulator, proved a considerable surprise. The subsequent measurement of bulk quantum oscillations in other correlated insulators including YbB$_{12}$ have lent support to our discovery of a class of unconventional insulators that are host to bulk quantum oscillations, of which SmB$_6$ was the first example. Here we perform a series of experiments to examine evidence for the intrinsic character of bulk quantum oscillations in floating zone-grown single crystals of SmB$_6$ that have been the subject of our quantum oscillation studies thus far. We present results of experiments including chemical composition analysis, magnetisation, thermal conductivity, electrical transport, and heat capacity on floating zone-grown single crystals of SmB$_6$, and a series of quantum oscillation experiments as a function of magnetic field, temperature, and magnetic field-orientation on single crystals of floating-zone grown SmB$_6$, LaB$_6$, and elemental Aluminium. Results of these experimental studies establish the intrinsic origin of quantum oscillations from the bulk of pristine floating zone-grown single crystals of SmB$_6$. The origin of the underlying bulk Fermi surface that bears close similarity with the unhybridised Fermi surface in metallic hexaborides despite the bulk insulating character of SmB$_6$ is thus at the heart of a theoretical mystery.