A connected, locally finite graph $Gamma$ is a Cayley--Abels graph for a totally disconnected, locally compact group $G$ if $G$ acts vertex-transitively with compact, open vertex stabilizers on $Gamma$. Define the minimal degree of $G$ as the minimal degree of a Cayley--Abels graph of $G$. We relate the minimal degree in various ways to the modular function, the scale function and the structure of compact open subgroups. As an application, we prove that if $T_d$ denotes the $d$-regular tree, then the minimal degree of ${rm Aut}(T_d)$ is $d$ for all $dgeq 2$.