We propose a novel scenario for possible electromagnetic (EM) emission by compact binary mergers in the accretion disks of active galactic nuclei (AGNs). Nuclear star clusters in AGNs are a plausible formation site of compact-stellar binaries (CSBs) whose coalescences can be detected through gravitational waves (GWs). We investigate the accretion onto and outflows from CSBs embedded in AGN disks. We show that these outflows are likely to create outflow cavities in the AGN disks before the binaries merge, which makes EM or neutrino counterparts much less common than would otherwise be expected. We discuss the necessary conditions for detectable EM counterparts to mergers inside the outflow cavities. If the merger remnant black hole experiences a high recoil velocity and can enter the AGN disk, it can accrete gas with a super-Eddington rate, newly forming a cavity-like structure. This bubble can break out of the disk within a day to a week after the merger. Such breakout emission can be bright enough to be detectable by current soft X-ray instruments, such as Swift-XRT and Chandra.