This paper presents $N$-body and stochastic models that describe the motion of tracer particles in a potential that contains a large population of extended substructures. Fluctuations of the gravitational field induce a random walk of orbital velocities that is fully specified by drift and diffusion coefficients. In the impulse and local approximations the coefficients are computed analytically from the number density, mass, size and relative velocity of substructures without arbitrary cuts in forces or impact parameters. The resulting Coulomb logarithm attains a well-defined geometrical meaning, $ln(Lambda)=ln (D/c)$, where $D/c$ is the ratio between the average separation and the individual size of substructures. Direct-force and Monte-Carlo $N$-body experiments show excellent agreement with the theory if substructures are sufficiently extended ($c/Dgtrsim 10^{-3}$) and not spatially overlapping ($c/Dlesssim 10^{-1}$). However, close encounters with point-like objects ($c/Dll 10^{-3}$) induce a heavy-tailed, non-Gaussian distribution of high-energy impulses that cannot be described with Brownian statistics. In the point-mass limit ($c/Dapprox 0$) the median Coulomb logarithm measured from $N$-body models deviates from the theoretical relation, converging towards a maximum value $langle ln(Lambda)rangle approx 8.2$ independently of the mass and relative velocity of nearby substructures.