Quantum processes of inherent dynamical nature, such as quantum walks (QWs), defy a description in terms of an equilibrium statistical physics ensemble. Up to now, it has remained a key challenge to identify general principles behind the underlying unitary quantum dynamics. Here, we show and experimentally observe that split-step QWs admit a characterization in terms of a dynamical topological order parameter (DTOP). This integer-quantized DTOP measures, at a given time, the winding of the geometric phase accumulated by the wave-function during the QW. We observe distinct dynamical regimes in our experimentally realized QWs each of which can be attributed to a qualitatively different temporal behavior of the DTOP. Upon identifying an equivalent many-body problem, we reveal an intriguing connection between the nonanalytic changes of the DTOP in QWs and the occurrence of dynamical quantum phase transitions.