Given a spherical homogeneous space G/H of minimal rank, we provide a simple procedure to describe its embeddings as varieties with torus action in terms of divisorial fans. The torus in question is obtained as the identity component of the quotient group N/H, where N is the normalizer of H in G. The resulting Chow quotient is equal to (a blowup of) the simple toroidal compactification of G/(H N^0). In the horospherical case, for example, it is equal to a flag variety, and the slices (coefficients) of the divisorial fan are merely shifts of the colored fan along the colors.