Thin films of Amorphous indium oxide undergo a magnetic field driven superconducting to insulator quantum phase transition. In the insulating phase, the current-voltage characteristics show large current discontinuities due to overheating of electrons. We show that the onset voltage for the discontinuities vanishes as we approach the quantum critical point. As a result the insulating phase becomes unstable with respect to any applied voltage making it, at least experimentally, immeasurable. We emphasize that unlike previous reports of the absence of linear response near quantum phase transitions, in our system, the departure from equilibrium is discontinuous. Because the conditions for these discontinuities are satisfied in most insulators at low temperatures, and due to the decay of all characteristic energy scales near quantum phase transitions, we believe that this instability is general and should occur in various systems while approaching their quantum critical point. Accounting for this instability is crucial for determining the critical behavior of systems near the transition.