We report some nonsmooth dynamics of a Bloch state in a one-dimensional tight binding model with the periodic boundary condition. After a sudden change of the potential of an arbitrary site, quantities like the survival probability of the particle in the initial Bloch state show cusps periodically, with the period being the Heisenberg time associated with the energy spectrum. This phenomenon is a emph{nonperturbative} counterpart of the nonsmooth dynamics observed previously (Zhang and Haque, arXiv:1404.4280) in a periodically driven tight binding model. Underlying the cusps is an exactly solvable model, which consists of equally spaced levels extending from $-infty$ to $+infty $, between which two arbitrary levels are coupled to each other by the same strength.