We show that twisted reduced models can be interpreted as noncommutative Yang-Mills theory. Based upon this correspondence, we obtain noncommutative Yang-Mills theory with D-brane backgrounds in IIB matrix model. We propose that IIB matrix model with D-brane backgrounds serve as a concrete definition of noncommutative Yang-Mills. We investigate D-instanton solutions as local excitations on D3-branes. When instantons overlap, their interaction can be well described in gauge theory and AdS/CFT correspondence. We show that IIB matrix model gives us the consistent potential with IIB supergravity when they are well separated.