Graph neural networks (GNNs) have recently achieved state-of-the-art performance in many graph-based applications. Despite the high expressive power, they typically need to perform an expensive recursive neighborhood expansion in multiple training ep
ochs and face a scalability issue. Moreover, most of them are inflexible since they are restricted to fixed-hop neighborhoods and insensitive to actual receptive field demands for different nodes. We circumvent these limitations by introducing a scalable and flexible Graph Attention Multilayer Perceptron (GAMLP). With the separation of the non-linear transformation and feature propagation, GAMLP significantly improves the scalability and efficiency by performing the propagation procedure in a pre-compute manner. With three principled receptive field attention, each node in GAMLP is flexible and adaptive in leveraging the propagated features over the different sizes of reception field. We conduct extensive evaluations on the three large open graph benchmarks (e.g., ogbn-papers100M, ogbn-products and ogbn-mag), demonstrating that GAMLP not only achieves the state-of-art performance, but also additionally provide high scalability and efficiency.
Emotion-cause pair extraction (ECPE), an emerging task in sentiment analysis, aims at extracting pairs of emotions and their corresponding causes in documents. This is a more challenging problem than emotion cause extraction (ECE), since it requires
no emotion signals which are demonstrated as an important role in the ECE task. Existing work follows a two-stage pipeline which identifies emotions and causes at the first step and pairs them at the second step. However, error propagation across steps and pair combining without contextual information limits the effectiveness. Therefore, we propose a Dual-Questioning Attention Network to alleviate these limitations. Specifically, we question candidate emotions and causes to the context independently through attention networks for a contextual and semantical answer. Also, we explore how weighted loss functions in controlling error propagation between steps. Empirical results show that our method performs better than baselines in terms of multiple evaluation metrics. The source code can be obtained at https://github.com/QixuanSun/DQAN.
High entropy alloys (HEAs) are a series of novel materials that demonstrate many exceptional mechanical properties. To understand the origin of these attractive properties, it is important to investigate the thermodynamics and elucidate the evolution
of various chemical phases. In this work, we introduce a data-driven approach to construct the effective Hamiltonian and study the thermodynamics of HEAs through canonical Monte Carlo simulation. The main characteristic of our method is to use pairwise interactions between atoms as features and systematically improve the representativeness of the dataset using samples from Monte Carlo simulation. We find this method produces highly robust and accurate effective Hamiltonians that give less than 0.1 mRy test error for all the three refractory HEAs: MoNbTaW, MoNbTaVW, and MoNbTaTiW. Using replica exchange to speed up the MC simulation, we calculated the specific heats and short-range order parameters in a wide range of temperatures. For all the studied materials, we find there are two major order-disorder transitions occurring respectively at $T_1$ and $T_2$, where $T_1$ is near room temperature but $T_2$ is much higher. We further demonstrate that the transition at $T_1$ is caused by W and Nb while the one at $T_2$ is caused by the other elements. By comparing with experiments, {color{black} the results provide insight into the role of chemical ordering in the strength and ductility of HEAs.
Semantic segmentation of 3D meshes is an important problem for 3D scene understanding. In this paper we revisit the classic multiview representation of 3D meshes and study several techniques that make them effective for 3D semantic segmentation of me
shes. Given a 3D mesh reconstructed from RGBD sensors, our method effectively chooses different virtual views of the 3D mesh and renders multiple 2D channels for training an effective 2D semantic segmentation model. Features from multiple per view predictions are finally fused on 3D mesh vertices to predict mesh semantic segmentation labels. Using the large scale indoor 3D semantic segmentation benchmark of ScanNet, we show that our virtual views enable more effective training of 2D semantic segmentation networks than previous multiview approaches. When the 2D per pixel predictions are aggregated on 3D surfaces, our virtual multiview fusion method is able to achieve significantly better 3D semantic segmentation results compared to all prior multiview approaches and competitive with recent 3D convolution approaches.
Interface structures in complex oxides remain one of the active areas of condensed matter physics research, largely enabled by recent advances in scanning transmission electron microscopy (STEM). Yet the nature of the STEM contrast in which the struc
ture is projected along the given direction precludes separation of possible structural models. Here, we utilize deep convolutional neural networks (DCNN) trained on simulated 4D scanning transmission electron microscopy (STEM) datasets to predict structural descriptors of interfaces. We focus on the widely studied interface between LaAlO3 and SrTiO3, using dynamical diffraction theory and leveraging high performance computing to simulate thousands of possible 4D STEM datasets to train the DCNN to learn properties of the underlying structures on which the simulations are based. We validate the DCNN on simulated data and show that it is possible (with >95% accuracy) to identify a physically rough from a chemically diffuse interface and achieve 85% accuracy in determination of buried step positions within the interface. The method shown here is general and can be applied for any inverse imaging problem where forward models are present.
We present a novel deep learning (DL) approach to produce highly accurate predictions of macroscopic physical properties of solid solution binary alloys and magnetic systems. The major idea is to make use of the correlations between different physica
l properties in alloy systems to improve the prediction accuracy of neural network (NN) models. We use multitasking NN models to simultaneously predict the total energy, charge density and magnetic moment. These physical properties mutually serve as constraints during the training of the multitasking NN, resulting in more reliable DL models because multiple physics properties are correctly learned by a single model. Two binary alloys, copper-gold (CuAu) and iron-platinum (FePt), were studied. Our results show that once the multitasking NNs are trained, they can estimate the material properties for a specific configuration hundreds of times faster than first-principles density functional theory calculations while retaining comparable accuracy. We used a simple measure based on the root-mean-squared errors (RMSE) to quantify the quality of the NN models, and found that the inclusion of charge density and magnetic moment as physical constraints leads to more stable models that exhibit improved accuracy and reduced uncertainty for the energy predictions.
Electron and nuclear spins of diamond nitrogen-vacancy (NV) centers are good candidates for quantum information processing as they have long coherence time and can be initialized and read out optically. However, creating a large number of coherently
coupled and individually addressable NV centers for quantum computing has been a big challenge. Here we propose methods to use high-density diamond NV centers coupled by spin-spin interaction with an average separation on the order of 10 nm for quantum computing. We propose to use a strain gradient to encode the position information of each NV center in the energy level of its excited electron orbital state, which causes a shift of its optical transition frequency. With such strain encoding, more than 100 closely-packed NV centers below optical diffraction limit can be read out individually by resonant optical excitation. A magnetic gradient will be used to shift the electron spin resonant (ESR) frequencies of NV centers. Therefore, the spin state of each NV center can be individually manipulated and different NV centers can be selectively coupled. A universal set of quantum operations for two-qubit and three-qubit system is introduced by careful design of external drives. Moreover, entangled states with multiple qubits can be created by this protocol, which is a major step towards quantum information processing with solid-state spins.
We introduce novel communication strategies in synchronous distributed Deep Learning consisting of decentralized gradient reduction orchestration and computational graph-aware grouping of gradient tensors. These new techniques produce an optimal over
lap between computation and communication and result in near-linear scaling (0.93) of distributed training up to 27,600 NVIDIA V100 GPUs on the Summit Supercomputer. We demonstrate our gradient reduction techniques in the context of training a Fully Convolutional Neural Network to approximate the solution of a longstanding scientific inverse problem in materials imaging. The efficient distributed training on a dataset size of 0.5 PB, produces a model capable of an atomically-accurate reconstruction of materials, and in the process reaching a peak performance of 2.15(4) EFLOPS$_{16}$.
We experimentally demonstrate that tomographic measurements can be performed for states of qubits before they are prepared. A variant of the quantum teleportation protocol is used as a channel between two instants in time, allowing measurements for p
olarisation states of photons to be implemented 88 ns before they are created. Measurement data taken at the early time and later unscrambled according to the results of the protocols Bell measurements, produces density matrices with an average fidelity of $0.90 pm 0.01$ against the ideal states of photons created at the later time. Process tomography of the time-reverse quantum channel finds an average process fidelity of $0.84 pm 0.02$. While our proof-of-principle implementation necessitates some post-selection, the general protocol is deterministic and requires no post-selection to sift desired states and reject a larger ensemble.
Face recognition performance improves rapidly with the recent deep learning technique developing and underlying large training dataset accumulating. In this paper, we report our observations on how big data impacts the recognition performance. Accord
ing to these observations, we build our Megvii Face Recognition System, which achieves 99.50% accuracy on the LFW benchmark, outperforming the previous state-of-the-art. Furthermore, we report the performance in a real-world security certification scenario. There still exists a clear gap between machine recognition and human performance. We summarize our experiments and present three challenges lying ahead in recent face recognition. And we indicate several possible solutions towards these challenges. We hope our work will stimulate the communitys discussion of the difference between research benchmark and real-world applications.
