Federated learning enables multiple clients to collaboratively learn a global model by periodically aggregating the clients models without transferring the local data. However, due to the heterogeneity of the system and data, many approaches suffer f
rom the client-drift issue that could significantly slow down the convergence of the global model training. As clients perform local updates on heterogeneous data through heterogeneous systems, their local models drift apart. To tackle this issue, one intuitive idea is to guide the local model training by the global teachers, i.e., past global models, where each client learns the global knowledge from past global models via adaptive knowledge distillation techniques. Coming from these insights, we propose a novel approach for heterogeneous federated learning, namely FedGKD, which fuses the knowledge from historical global models for local training to alleviate the client-drift issue. In this paper, we evaluate FedGKD with extensive experiments on various CV/NLP datasets (i.e., CIFAR-10/100, Tiny-ImageNet, AG News, SST5) and different heterogeneous settings. The proposed method is guaranteed to converge under common assumptions, and achieves superior empirical accuracy in fewer communication runs than five state-of-the-art methods.
In this paper, we introduce a new acoustic leakage dataset of gas pipelines, called as GPLA-12, which has 12 categories over 684 training/testing acoustic signals. Unlike massive image and voice datasets, there have relatively few acoustic signal dat
asets, especially for engineering fault detection. In order to enhance the development of fault diagnosis, we collect acoustic leakage signals on the basis of an intact gas pipe system with external artificial leakages, and then preprocess the collected data with structured tailoring which are turned into GPLA-12. GPLA-12 dedicates to serve as a feature learning dataset for time-series tasks and classifications. To further understand the dataset, we train both shadow and deep learning algorithms to observe the performance. The dataset as well as the pretrained models have been released at both www.daip.club and github.com/Deep-AI-Application-DAIP
For many data mining and machine learning tasks, the quality of a similarity measure is the key for their performance. To automatically find a good similarity measure from datasets, metric learning and similarity learning are proposed and studied ext
ensively. Metric learning will learn a Mahalanobis distance based on positive semi-definite (PSD) matrix, to measure the distances between objectives, while similarity learning aims to directly learn a similarity function without PSD constraint so that it is more attractive. Most of the existing similarity learning algorithms are online similarity learning method, since online learning is more scalable than offline learning. However, most existing online similarity learning algorithms learn a full matrix with d 2 parameters, where d is the dimension of the instances. This is clearly inefficient for high dimensional tasks due to its high memory and computational complexity. To solve this issue, we introduce several Sparse Online Relative Similarity (SORS) learning algorithms, which learn a sparse model during the learning process, so that the memory and computational cost can be significantly reduced. We theoretically analyze the proposed algorithms, and evaluate them on some real-world high dimensional datasets. Encouraging empirical results demonstrate the advantages of our approach in terms of efficiency and efficacy.
Antiferromagnetism (AF) such as Neel ordering is often closely related to Coulomb interactions such as Hubbard repulsion in two-dimensional (2D) systems. Whether Neel AF ordering in 2D can be dominantly induced by electron-phonon couplings (EPC) has
not been completely understood. Here, by employing numerically-exact sign-problem-free quantum Monte Carlo (QMC) simulations, we show that optical Su-Schrieffer-Heeger (SSH) phonons with frequency $omega$ and EPC constant $lambda$ can induce AF ordering for a wide range of phonon frequency $omega>omega_c$. For $omega<omega_c$, a valence-bond-solid (VBS) order appears and there is a direct quantum phase transition between VBS and AF phases at $omega_c$. The phonon mechanism of the AF ordering is related to the fact that SSH phonons directly couple to electron hopping whose second-order process can induce an effective AF spin exchange. Our results shall shed new lights to understanding AF ordering in correlated quantum materials.
We analyze the quantum phase diagram of the Holstein-Hubbard model using an asymptotically exact strong-coupling expansion. We find all sorts of interesting phases including a pair-density wave (PDW), a charge 4e (and even a charge 6e) superconductor
, regimes of phase separation, and a variety of distinct charge-density-wave (CDW), spin-density-wave (SDW) and superconducting regimes. We chart the crossovers that occur as a function of the degree of retardation, i.e. the ratio of characteristic phonon frequency to the strength of interactions.
Protein-protein interaction (PPI) extraction from published scientific literature provides additional support for precision medicine efforts. Meanwhile, knowledge bases (KBs) contain huge amounts of structured information of protein entities and thei
r relations, which can be encoded in entity and relation embeddings to help PPI extraction. However, the prior knowledge of protein-protein pairs must be selectively used so that it is suitable for different contexts. This paper proposes a Knowledge Selection Model (KSM) to fuse the selected prior knowledge and context information for PPI extraction. Firstly, two Transformers encode the context sequence of a protein pair according to each protein embedding, respectively. Then, the two outputs are fed to a mutual attention to capture the important context features towards the protein pair. Next, the context features are used to distill the relation embedding by a knowledge selector. Finally, the selected relation embedding and the context features are concatenated for PPI extraction. Experiments on the BioCreative VI PPI dataset show that KSM achieves a new state-of-the-art performance (38.08% F1-score) by adding knowledge selection.
According to Landau criterion, a phase transition should be first order when cubic terms of order parameters are allowed in its effective Ginzburg-Landau free energy. Recently, it was shown by renormalization group (RG) analysis that continuous trans
ition can happen at putatively first-order $Z_3$ transitions in 2D Dirac semimetals and such non-Landau phase transitions were dubbed fermion-induced quantum critical points (FIQCP) [Li et al., Nature Communications 8, 314 (2017)]. The RG analysis, controlled by the 1/$N$ expansion with $N$ the number of flavors of four-component Dirac fermions, shows that FIQCP occurs for $Ngeq N_c$. Previous QMC simulations of a microscopic model of SU($N$) fermions on the honeycomb lattice showed that FIQCP occurs at the transition between Dirac semimetals and Kekule-VBS for $Ngeq 2$. However, precise value of the lower bound $N_c$ has not been established. Especially, the case of $N=1$ has not been explored by studying microscopic models so far. Here, by introducing a generalized SU($N$) fermion model with $N=1$ (namely spinless fermions on the honeycomb lattice), we perform large-scale sign-problem-free Majorana quantum Monte Carlo simulations and find convincing evidence of FIQCP for $N=1$. Consequently, our results suggest that FIQCP can occur in 2D Dirac semimetals for all positive integers $Ngeq 1$.
Many aspects of many-body localization (MBL), including dynamic classification of MBL phases, remain elusive. Here, by performing real-space renormalization group (RSRG) analysis we propose that there are two distinct types of MBL phases: strong MBL
induced by quasiperiodic (QP) potential and weak MBL induced by random potential. Strong and weak MBL phases can be distinguished by their different probability distributions of thermal inclusion and entanglement entropy: exponential decay in strong MBL phases but power-law decay in weak MBL. We further discuss underlying mechanisms as well as experimental implications of having two distinct types of MBL phases. Strong MBL induced by QP potential may provide a more robust and promising platform for quantum information storage and processing.
Exotic physics often emerges around quantum criticality in metallic systems. Here we explore the nature of topological phase transitions between 3D double-Weyl semimetals and insulators (through annihilating double-Weyl nodes with opposite chiralitie
s) in the presence of Coulomb interactions. From renormalization-group (RG) analysis, we find a non-Fermi-liquid quantum critical point (QCP) between the double-Weyl semimetals and insulators when artificially neglecting short-range interactions. However, it is shown that this non-Fermi-liquid QCP is actually unstable against nematic ordering when short-range interactions are correctly included in the RG analysis. In other words, the putative QCP between the semimetals and insulators is preempted by emergence of nematic phases when Coulomb interactions are present. We further discuss possible experimental relevance of the nematicity-preempted QCP to double-Weyl candidate materials HgCr2Se4 and SrSi2.
Precise nature of MBL transitions in both random and quasiperiodic (QP) systems remains elusive so far. In particular, whether MBL transitions in QP and random systems belong to the same universality class or two distinct ones has not been decisively
resolved. Here we investigate MBL transitions in one-dimensional ($d!=!1$) QP systems as well as in random systems by state-of-the-art real-space renormalization group (RG) calculation. Our real-space RG shows that MBL transitions in 1D QP systems are characterized by the critical exponent $ u!approx!2.4$, which respects the Harris-Luck bound ($ u!>!1/d$) for QP systems. Note that $ u!approx! 2.4$ for QP systems also satisfies the Harris-CCFS bound ($ u!>!2/d$) for random systems, which implies that MBL transitions in 1D QP systems are stable against weak quenched disorder since randomness is Harris irrelevant at the transition. We shall briefly discuss experimental means to measure $ u$ of QP-induced MBL transitions.
