General relativity is a fully conservative theory, but there exist other possible metric theories of gravity. We consider non-conservative ones with a parameterized post-Newtonian (PPN) parameter, $zeta_2$. A non-zero $zeta_2$ induces a self-acceleration for the center of mass of an eccentric binary pulsar system, which contributes to the second time derivative of the pulsar spin frequency, $ddot{ u}$. In our work, using the method in Will (1992), we provide an improved analysis with four well-timed, carefully-chosen binary pulsars. In addition, we extend Wills method and derive $zeta_2$s effect on the third time derivative of the spin frequency, $dddot{ u}$. For PSR B1913+16, the constraint from $dddot{ u}$ is even tighter than that from $ddot{ u}$. We combine multiple pulsars with Bayesian inference, and obtain an upper limit, $left|zeta_{2}right|<1.3times10^{-5}$ at 95% confidence level, assuming a flat prior in $log_{10} left| zeta_{2}right|$. It improves the existing bound by a factor of three. Moreover, we propose an analytical timing formalism for $zeta_2$. Our simulated times of arrival with simplified assumptions show binary pulsars capability in limiting $zeta_{2}$, and useful clues are extracted for real data analysis in future. In particular, we discover that for PSRs B1913+16 and J0737$-$3039A, $dddot{ u}$ can yield more constraining limits than $ddot{ u}$.