An Efficient Mini-batch Method via Partial Transportation

Mini-batch optimal transport (m-OT) has been widely used recently to deal with the memory issue of OT in large-scale applications. Despite their practicality, m-OT suffers from misspecified mappings, namely, mappings that are optimal on the mini-batch level but do not exist in the optimal transportation plan between the original measures. To address the misspecified mappings issue, we propose a novel mini-batch method by using partial optimal transport (POT) between mini-batch empirical measures, which we refer to as mini-batch partial optimal transport (m-POT). Leveraging the insight from the partial transportation, we explain the source of misspecified mappings from the m-OT and motivate why limiting the amount of transported masses among mini-batches via POT can alleviate the incorrect mappings. Finally, we carry out extensive experiments on various applications to compare m-POT with m-OT and recently proposed mini-batch method, mini-batch unbalanced optimal transport (m-UOT). We observe that m-POT is better than m-OT deep domain adaptation applications while having comparable performance with m-UOT. On other applications, such as deep generative model, gradient flow, and color transfer, m-POT yields more favorable performance than both m-OT and m-UOT.

Performance Improvement of LoRa Modulation with Signal Combining and Semi-Coherent Detection

In this paper, we investigate performance improvements of low-power long-range (LoRa) modulation when a gateway is equipped with multiple antennas. We derive the optimal decision rules for both coherent and non-coherent detections when combining signals received from multiple antennas. To provide insights on how signal combining can benefit LoRa systems, we present expressions of the symbol/bit error probabilities of both the coherent and non-coherent detections in AWGN and Rayleigh fading channels, respectively. Moreover, we also propose an iterative semi-coherent detection that does not require any overhead to estimate the channel-state-information (CSI) while its performance can approach that of the ideal coherent detection. Simulation and analytical results show very large power gains, or coverage extension, provided by the use of multiple antennas for all the detection schemes considered.

Structured Dropout Variational Inference for Bayesian Neural Networks

Approximate inference in deep Bayesian networks exhibits a dilemma of how to yield high fidelity posterior approximations while maintaining computational efficiency and scalability. We tackle this challenge by introducing a novel variational structured approximation inspired by the Bayesian interpretation of Dropout regularization. Concretely, we focus on the inflexibility of the factorized structure in Dropout posterior and then propose an improved method called Variational Structured Dropout (VSD). VSD employs an orthogonal transformation to learn a structured representation on the variational noise and consequently induces statistical dependencies in the approximate posterior. Theoretically, VSD successfully addresses the pathologies of previous Variational Dropout methods and thus offers a standard Bayesian justification. We further show that VSD induces an adaptive regularization term with several desirable properties which contribute to better generalization. Finally, we conduct extensive experiments on standard benchmarks to demonstrate the effectiveness of VSD over state-of-the-art variational methods on predictive accuracy, uncertainty estimation, and out-of-distribution detection.

BoMb-OT: On Batch of Mini-batches Optimal Transport

Mini-batch optimal transport (m-OT) has been successfully used in practical applications that involve probability measures with intractable density, or probability measures with a very high number of supports. The m-OT solves several sparser optimal transport problems and then returns the average of their costs and transportation plans. Despite its scalability advantage, the m-OT does not consider the relationship between mini-batches which leads to undesirable estimation. Moreover, the m-OT does not approximate a proper metric between probability measures since the identity property is not satisfied. To address these problems, we propose a novel mini-batching scheme for optimal transport, named Batch of Mini-batches Optimal Transport (BoMb-OT), that finds the optimal coupling between mini-batches and it can be seen as an approximation to a well-defined distance on the space of probability measures. Furthermore, we show that the m-OT is a limit of the entropic regularized version of the BoMb-OT when the regularized parameter goes to infinity. Finally, we carry out extensive experiments to show that the BoMb-OT can estimate a better transportation plan between two original measures than the m-OT. It leads to a favorable performance of the BoMb-OT in the matching and color transfer tasks. Furthermore, we observe that the BoMb-OT also provides a better objective loss than the m-OT for doing approximate Bayesian computation, estimating parameters of interest in parametric generative models, and learning non-parametric generative models with gradient flow.

Improving Relational Regularized Autoencoders with Spherical Sliced Fused Gromov Wasserstein

Relational regularized autoencoder (RAE) is a framework to learn the distribution of data by minimizing a reconstruction loss together with a relational regularization on the latent space. A recent attempt to reduce the inner discrepancy between the prior and aggregated posterior distributions is to incorporate sliced fused Gromov-Wasserstein (SFG) between these distributions. That approach has a weakness since it treats every slicing direction similarly, meanwhile several directions are not useful for the discriminative task. To improve the discrepancy and consequently the relational regularization, we propose a new relational discrepancy, named spherical sliced fused Gromov Wasserstein (SSFG), that can find an important area of projections characterized by a von Mises-Fisher distribution. Then, we introduce two variants of SSFG to improve its performance. The first variant, named mixture spherical sliced fused Gromov Wasserstein (MSSFG), replaces the vMF distribution by a mixture of von Mises-Fisher distributions to capture multiple important areas of directions that are far from each other. The second variant, named power spherical sliced fused Gromov Wasserstein (PSSFG), replaces the vMF distribution by a power spherical distribution to improve the sampling time in high dimension settings. We then apply the new discrepancies to the RAE framework to achieve its new variants. Finally, we conduct extensive experiments to show that the new proposed autoencoders have favorable performance in learning latent manifold structure, image generation, and reconstruction.

Pulsar Timing Array Constraints on the Merger Timescale of Subparsec Supermassive Black Hole Binary Candidates

We estimate the merger timescale of spectroscopically-selected, subparsec supermassive black hole binary (SMBHB) candidates by comparing their expected contribution to the gravitational wave background (GWB) with the sensitivity of current pulsar timing array (PTA) experiments and in particular, with the latest upper limit placed by the North American Nanohertz Observatory for Gravitational Waves (NANOGrav). We find that the average timescale to coalescence of such SMBHBs is $langle t_{rm evol} rangle > 6times 10^4,$yr, assuming that their orbital evolution in the PTA frequency band is driven by emission of gravitational waves. If some fraction of SMBHBs do not reside in spectroscopically detected active galaxies, and their incidence in active and inactive galaxies is similar, then the merger timescale could be $sim 10$ times longer, $langle t_{rm evol} rangle > 6times 10^5,$yr. These limits are consistent with the range of timescales predicted by theoretical models and imply that all the SMBHB candidates in our spectroscopic sample could be binaries without violating the observational constraints on the GWB. This result illustrates the power of the multi-messenger approach, facilitated by the PTAs, in providing an independent statistical test of the nature of SMBHB candidates discovered in electromagnetic searches.

Distributional Sliced-Wasserstein and Applications to Generative Modeling

Sliced-Wasserstein distance (SW) and its variant, Max Sliced-Wasserstein distance (Max-SW), have been used widely in the recent years due to their fast computation and scalability even when the probability measures lie in a very high dimensional space. However, SW requires many unnecessary projection samples to approximate its value while Max-SW only uses the most important projection, which ignores the information of other useful directions. In order to account for these weaknesses, we propose a novel distance, named Distributional Sliced-Wasserstein distance (DSW), that finds an optimal distribution over projections that can balance between exploring distinctive projecting directions and the informativeness of projections themselves. We show that the DSW is a generalization of Max-SW, and it can be computed efficiently by searching for the optimal push-forward measure over a set of probability measures over the unit sphere satisfying certain regularizing constraints that favor distinct directions. Finally, we conduct extensive experiments with large-scale datasets to demonstrate the favorable performances of the proposed distances over the previous sliced-based distances in generative modeling applications.

Emission Signatures from Sub-parsec Binary Supermassive Black Holes III: Comparison of Models with Observations

We present a method for comparing the H$beta$ emission-line profiles of observed supermassive black hole (SBHB) candidates and models of sub-parsec SBHBs in circumbinary disks. Using the approach based on principal component analysis we infer the values of the binary parameters for the spectroscopic SBHB candidates and evaluate the parameter degeneracies, representative of the uncertainties intrinsic to such measurements. We find that as a population, the SBHB candidates favor the average value of the semimajor axis corresponding to $log(a/M) approx 4.20pm 0.42$ and comparable mass ratios, $q>0.5$. If the SBHB candidates considered are true binaries, this result would suggest that there is a physical process that allows initially unequal mass systems to evolve toward comparable mass ratios (e.g., accretion that occurs preferentially onto the smaller of the black holes) or point to some, yet unspecified, selection bias. Our method also indicates that the SBHB candidates equally favor configurations in which the mini-disks are coplanar or misaligned with the binary orbital plane. If confirmed for true SBHBs, this finding would indicate the presence of a physical mechanism that maintains misalignment of the mini-disks down to sub-parsec binary separations (e.g., precession driven by gravitational torques). The probability distributions of the SBHB parameters inferred for the observed SBHB candidates and our control group of AGNs are statistically indistinguishable, implying that this method can in principle be used to interpret the observed emission-line profiles once a sample of confirmed SBHBs is available but cannot be used as a conclusive test of binarity.

Emission Signatures from Sub-parsec Binary Supermassive Black Holes II: Effect of Accretion Disk Wind on Broad Emission Lines

We present an improved semi-analytic model for calculation of the broad optical emission-line signatures from sub-parsec supermassive black hole binaries (SBHBs) in circumbinary disks. The second-generation model improves upon the treatment of radiative transfer by taking into account the effect of the radiation driven accretion disk wind on the properties of the emission-line profiles. Analysis of 42.5 million modeled emission-line profiles shows that correlations between the profile properties and SBHB parameters identified in the first-generation model are preserved, indicating that their diagnostic power is not diminished. The profile shapes are a more sensitive measure of the binary orbital separation and the degree of alignment of the black hole mini-disks, and are less sensitive to the SBHB mass ratio and orbital eccentricity. We also find that modeled profile shapes are more compatible with the observed sample of SBHB candidates than with our control sample of regular AGNs. Furthermore, if the observed sample of SBHBs is made up of genuine binaries, it must include compact systems with comparable masses, and misaligned mini-disks. We note that the model described in this paper can be used to interpret the observed emission-line profiles once a sample of confirmed SBHBs is available but cannot be used to prove that the observed SBHB candidates are true binaries.

Likelihood for Detection of Sub-parsec Supermassive Black Hole Binaries in Spectroscopic Surveys

Motivated by observational searches for sub-parsec supermassive black hole binaries (SBHBs) we develop a modular analytic model to determine the likelihood for detection of SBHBs by ongoing spectroscopic surveys. The model combines the parametrized rate of orbital evolution of SBHBs in circumbinary disks with the selection effects of spectroscopic surveys and returns a multivariate likelihood for SBHB detection. Based on this model we find that in order to evolve into the detection window of the spectroscopic searches from larger separations in less than a Hubble time, $10^8M_odot$ SBHBs must, on average, experience angular momentum transport faster than that provided by a disk with accretion rate $0.06,dot{M}_E$. Spectroscopic searches with yearly cadence of observations are in principle sensitive to binaries with orbital separations $< {rm few}times 10^4, r_g$ ($r_g = GM/c^2$ and $M$ is the binary mass), and for every one SBHB in this range there should be over 200 more gravitationally bound systems with similar properties, at larger separations. Furthermore, if spectra of all SBHBs in this separation range exhibit the AGN-like emission lines utilized by spectroscopic searches, the projection factors imply five undetected binaries for each observed $10^8M_odot$ SBHB with mass ratio $0.3$ and orbital separation $10^4,r_g$ (and more if some fraction of SBHBs is inactive). This model can be used to infer the most likely orbital parameters for observed SBHB candidates and to provide constraints on the rate of orbital evolution of SBHBs, if observed candidates are shown to be genuine binaries.