Given a grading $Gamma: A=oplus_{gin G}A_g$ on a nonassociative algebra $A$ by an abelian group $G$, we have two subgroups of the group of automorphisms of $A$: the automorphisms that stabilize each homogeneous component $A_g$ (as a subspace) and the automorphisms that permute the components. By the Weyl group of $Gamma$ we mean the quotient of the latter subgroup by the former. In the case of a Cartan decomposition of a semisimple complex Lie algebra, this is the automorphism group of the root system, i.e., the so-called extended Weyl group. A grading is called fine if it cannot be refined. We compute the Weyl groups of all fine gradings on matrix algebras, octonions and the Albert algebra over an algebraically closed field (of characteristic different from 2 in the case of the Albert algebra).