Let $J_G$ be the binomial edge ideal of a graph $G$. We characterize all graphs whose binomial edge ideals, as well as their initial ideals, have regularity $3$. Consequently we characterize all graphs $G$ such that $J_G$ is extremal Gorenstein. Indeed, these characterizations are consequences of an explicit formula we obtain for the regularity of the binomial edge ideal of the join product of two graphs. Finally, by using our regularity formula, we discuss some open problems in the literature. In particular we disprove a conjecture in cite{CDI} on the regularity of weakly closed graphs.