Interlayer transport in high-$T_C$ cuprates is mediated by superconducting tunneling across the CuO$_2$ planes. For this reason, the terahertz frequency optical response is dominated by one or more Josephson plasma resonances and becomes highly nonlinear at fields for which the tunneling supercurrents approach their critical value, $I_C$. These large terahertz nonlinearities are in fact a hallmark of superconducting transport. Surprisingly, however, they have been documented in La$_{2-x}$Ba$_x$CuO$_4$ also above $T_C$ for doping values near $x=1/8$, and interpreted as an indication of superfluidity in the stripe phase. Here, Electric Field Induced Second Harmonic (EFISH) is used to study the dynamics of time-dependent interlayer voltages when La$_{2-x}$Ba$_x$CuO$_4$ is driven with large-amplitude terahertz pulses, in search of other characteristic signatures of Josephson tunnelling in the normal state. We show that this method is sensitive to the voltage anomalies associated with 2$pi$ Josephson phase slips, which near $x=1/8$ are observed both below and above $T_C$. These results document a new regime of nonlinear transport that shares features of sliding charge-density-waves and superconducting phase dynamics.