The out-of-time-ordered correlators (OTOC) have been established as a fundamental concept for quantifying quantum information scrambling and diagnosing quantum chaotic behavior. Recently, it was theoretically proposed that the OTOC can be used as an order parameter to dynamically detect both equilibrium quantum phase transitions (EQPTs) and dynamical quantum phase transitions (DQPTs) in one-dimensional many-body systems. Here we report the first experimental observation of EQPTs and DQPTs in a quantum spin chain via quench dynamics of OTOC on a nuclear magnetic resonance quantum simulator. We observe that the quench dynamics of both the order parameter and the two-body correlation function cannot detect the DQPTs, but the OTOC can unambiguously detect the DQPTs. Moreover, we demonstrate that the long-time average value of the OTOC in quantum quench signals the equilibrium quantum critical point and ordered quantum phases, thus one can measure the EQPTs from the non-equilibrium quantum quench dynamics. Our experiment paves a way for experimentally investigating DQPTs through OTOCs and for studying the EQPTs through the non-equilibrium quantum quench dynamics with quantum simulators.