In this article, we construct countably many mutually non-isotopic diffeomorphisms of some closed non simply-connected 4-manifolds that are homotopic to but not isotopic to the identity, by surgery along $Theta$-graphs. As corollaries of this, we obtain some new results on codimension 1 embeddings and pseudo-isotopies of 4-manifolds. In the proof of the non-triviality of the diffeomorphisms, we utilize a twisted analogue of Kontsevichs characteristic class for smooth bundles, which is obtained by extending a higher dimensional analogue of March{e}--Lescops equivariant triple intersection in configuration spaces of 3-manifolds to allow Lie algebraic local coefficient system.