Observations of HDO are an important complement for studies of water, because they give strong constraints on the formation processes -- grain surfaces versus energetic process in the gas phase, e.g. in shocks. The HIFI observations of multiple transitions of HDO in Sgr~B2(M) presented here allow the determination of the HDO abundance throughout the envelope, which has not been possible before with ground-based observations only. The abundance structure has been modeled with the spherical Monte Carlo radiative transfer code RATRAN, which also takes radiative pumping by continuum emission from dust into account. The modeling reveals that the abundance of HDO rises steeply with temperature from a low abundance ($2.5times 10^{-11}$) in the outer envelope at temperatures below 100~K through a medium abundance ($1.5times 10^{-9}$) in the inner envelope/outer core, at temperatures between 100 and 200~K, and finally a high abundance ($3.5times 10^{-9}$) at temperatures above 200~K in the hot core.