For a classical link, Milnor defined a family of isotopy invariants, called Milnor $overline{mu}$-invariants. Recently, Chrisman extended Milnor $overline{mu}$-invariants to welded links by a topological approach. The aim of this paper is to show that Milnor $overline{mu}$-invariants can be extended to welded links by a combinatorial approach. The proof contains an alternative proof for the invariance of the original $overline{mu}$-invariants of classical links.