We explore the nonlinear dynamics of a cavity optomechanical system. Our realization consisting of a drumhead nano-electro-mechanical resonator (NEMS) coupled to a microwave cavity, allows for a nearly ideal platform to study the nonlinearities arising purely due to radiation-pressure physics. Experiments are performed under a strong microwave Stokes pumping which triggers mechanical self-sustained oscillations. We analyze the results in the framework of an extended nonlinear optomechanical theory, and demonstrate that quadratic and cubic coupling terms in the opto-mechanical Hamiltonian have to be considered. Quantitative agreement with the measurements is obtained considering only genuine geometrical nonlinearities: no thermo-optical instabilities are observed, in contrast with laser-driven systems. Based on these results, we describe a method to quantify nonlinear properties of microwave optomechanical devices. Such a technique, available now in the quantum electro-mechanics toolbox, but completely generic, is mandatory for the development of new schemes where higher-order coupling terms are proposed as a new resource, like Quantum Non-Demolition measurements, or in the search for new fundamental quantum signatures, like Quantum Gravity. We also find that the motion imprints a wide comb of extremely narrow peaks in the microwave output field, which could also be exploited in specific microwave-based measurements, potentially limited only by the quantum noise of the optical and the mechanical fields for a ground-state cooled NEMS device.
Download