The vertex operators for the supergravity multiplet can be constructed through the Wilson lines in IIB matrix model. We investigate the structure of the vertex operators and the symmetry of their correlation functions. For this purpose, we perturb the theory by the Wilson lines dual to the supergravity multiplet. The structure of the Wilson lines can be determined by requiring $cal{N}$=2 SUSY under the low energy approximation. We argue that the generating functional of the correlators is invariant under local SUSY transformations of IIB supergravity.