The topological vertex formalism for 5d $mathcal{N}=1$ gauge theories is not only a convenient tool to compute the instanton partition function of these theories, but it is also accompanied by a nice algebraic structure that reveals various kinds of nice properties such as dualities and integrability of the underlying theories. The usual refined topological vertex formalism is derived for gauge theories with $A$-type quiver structure (and $A$-type gauge groups). In this article, we propose a construction with a web of vertex operators for all $ABCDEFG$-type and affine quivers by introducing several new vertices into the formalism, based on the reproducing of known instanton partition functions and qq-characters for these theories.