We investigate low-density accretion flows onto massive black holes (BHs) with masses of $gtrsim 10^5~M_odot$ orbiting around in the outskirts of their host galaxies, performing three-dimensional hydrodynamical simulations. Those wandering BHs are populated via ejection from the galactic nuclei through multi-body BH interactions and gravitational wave recoils associated with galaxy and BH coalescences. We find that when a wandering BH is fed with hot and diffuse plasma with density fluctuations, the mass accretion rate is limited at $sim 10-20%$ of the canonical Bondi-Hoyle-Littleton rate owing to a wide distribution of inflowing angular momentum. We further calculate radiation spectra from radiatively inefficient accretion flows onto the wandering BH using a semi-analytical two-temperature disk model and find that the predicted spectra have a peak at the millimeter band, where the Atacama Large Millimeter/submillimeter Array (ALMA) has the highest sensitivity and spatial resolution. Millimeter observations with ALMA and future facilities such as the next generation Very Large Array (ngVLA) will enable us to hunt for a population of wandering BHs and push the detectable mass limit down to $M_bullet simeq 2times10^7~M_odot$ for massive nearby ellipticals, e.g., M87, and $M_bullet simeq 10^5~M_odot$ for the Milky Way. This radiation spectral model, combined with numerical simulations, will be applied to give physical interpretations of off-nuclear BHs detected in dwarf galaxies, which may constrain BH seed formation scenarios.