In this paper, we analyse a new exponential-type integrator for the nonlinear cubic Schrodinger equation on the $d$ dimensional torus $mathbb T^d$. The scheme has recently also been derived in a wider context of decorated trees in [Y. Bruned and K. S
chratz, arXiv:2005.01649]. It is explicit and efficient to implement. Here, we present an alternative derivation, and we give a rigorous error analysis. In particular, we prove second-order convergence in $H^gamma(mathbb T^d)$ for initial data in $H^{gamma+2}(mathbb T^d)$ for any $gamma > d/2$. This improves the previous work in [Knoller, A. Ostermann, and K. Schratz, SIAM J. Numer. Anal. 57 (2019), 1967-1986]. The design of the scheme is based on a new method to approximate the nonlinear frequency interaction. This allows us to deal with the complex resonance structure in arbitrary dimensions. Numerical experiments that are in line with the theoretical result complement this work.
Predicting the next interaction of a short-term interaction session is a challenging task in session-based recommendation. Almost all existing works rely on item transition patterns, and neglect the impact of user historical sessions while modeling u
ser preference, which often leads to non-personalized recommendation. Additionally, existing personalized session-based recommenders capture user preference only based on the sessions of the current user, but ignore the useful item-transition patterns from other users historical sessions. To address these issues, we propose a novel Heterogeneous Global Graph Neural Networks (HG-GNN) to exploit the item transitions over all sessions in a subtle manner for better inferring user preference from the current and historical sessions. To effectively exploit the item transitions over all sessions from users, we propose a novel heterogeneous global graph that contains item transitions of sessions, user-item interactions and global co-occurrence items. Moreover, to capture user preference from sessions comprehensively, we propose to learn two levels of user representations from the global graph via two graph augmented preference encoders. Specifically, we design a novel heterogeneous graph neural network (HGNN) on the heterogeneous global graph to learn the long-term user preference and item representations with rich semantics. Based on the HGNN, we propose the Current Preference Encoder and the Historical Preference Encoder to capture the different levels of user preference from the current and historical sessions, respectively. To achieve personalized recommendation, we integrate the representations of the user current preference and historical interests to generate the final user preference representation. Extensive experimental results on three real-world datasets show that our model outperforms other state-of-the-art methods.
Electrically interfacing atomically thin transition metal dichalcogenide semiconductors (TMDSCs) with metal leads is challenging because of undesired interface barriers, which have drastically constrained the electrical performance of TMDSC devices f
or exploring their unconventional physical properties and realizing potential electronic applications. Here we demonstrate a strategy to achieve nearly barrier-free electrical contacts with few-layer TMDSCs by engineering interfacial bonding distortion. The carrier-injection efficiency of such electrical junction is substantially increased with robust ohmic behaviors from room to cryogenic temperatures. The performance enhancements of TMDSC field-effect transistors are well reflected by the ultralow contact resistance (down to 90 Ohm um in MoS2, towards the quantum limit), the ultrahigh field-effect mobility (up to 358,000 cm2V-1s-1 in WSe2) and the prominent transport characteristics at cryogenic temperatures. This method also offers new possibilities of the local manipulation of structures and electronic properties for TMDSC device design.
Deep learning has become the dominant approach in coping with various tasks in Natural LanguageProcessing (NLP). Although text inputs are typically represented as a sequence of tokens, there isa rich variety of NLP problems that can be best expressed
with a graph structure. As a result, thereis a surge of interests in developing new deep learning techniques on graphs for a large numberof NLP tasks. In this survey, we present a comprehensive overview onGraph Neural Networks(GNNs) for Natural Language Processing. We propose a new taxonomy of GNNs for NLP, whichsystematically organizes existing research of GNNs for NLP along three axes: graph construction,graph representation learning, and graph based encoder-decoder models. We further introducea large number of NLP applications that are exploiting the power of GNNs and summarize thecorresponding benchmark datasets, evaluation metrics, and open-source codes. Finally, we discussvarious outstanding challenges for making the full use of GNNs for NLP as well as future researchdirections. To the best of our knowledge, this is the first comprehensive overview of Graph NeuralNetworks for Natural Language Processing.
Graph neural networks (GNN) have been successful in many fields, and derived various researches and applications in real industries. However, in some privacy sensitive scenarios (like finance, healthcare), training a GNN model centrally faces challen
ges due to the distributed data silos. Federated learning (FL) is a an emerging technique that can collaboratively train a shared model while keeping the data decentralized, which is a rational solution for distributed GNN training. We term it as federated graph learning (FGL). Although FGL has received increasing attention recently, the definition and challenges of FGL is still up in the air. In this position paper, we present a categorization to clarify it. Considering how graph data are distributed among clients, we propose four types of FGL: inter-graph FL, intra-graph FL and graph-structured FL, where intra-graph is further divided into horizontal and vertical FGL. For each type of FGL, we make a detailed discussion about the formulation and applications, and propose some potential challenges.
Generating videos from text is a challenging task due to its high computational requirements for training and infinite possible answers for evaluation. Existing works typically experiment on simple or small datasets, where the generalization ability
is quite limited. In this work, we propose GODIVA, an open-domain text-to-video pretrained model that can generate videos from text in an auto-regressive manner using a three-dimensional sparse attention mechanism. We pretrain our model on Howto100M, a large-scale text-video dataset that contains more than 136 million text-video pairs. Experiments show that GODIVA not only can be fine-tuned on downstream video generation tasks, but also has a good zero-shot capability on unseen texts. We also propose a new metric called Relative Matching (RM) to automatically evaluate the video generation quality. Several challenges are listed and discussed as future work.
Contrastive Learning has emerged as a powerful representation learning method and facilitates various downstream tasks especially when supervised data is limited. How to construct efficient contrastive samples through data augmentation is key to its
success. Unlike vision tasks, the data augmentation method for contrastive learning has not been investigated sufficiently in language tasks. In this paper, we propose a novel approach to construct contrastive samples for language tasks using text summarization. We use these samples for supervised contrastive learning to gain better text representations which greatly benefit text classification tasks with limited annotations. To further improve the method, we mix up samples from different classes and add an extra regularization, named Mixsum, in addition to the cross-entropy-loss. Experiments on real-world text classification datasets (Amazon-5, Yelp-5, AG News, and IMDb) demonstrate the effectiveness of the proposed contrastive learning framework with summarization-based data augmentation and Mixsum regularization.
While substantial scholarship has focused on estimating the susceptibility of jobs to automation, little has examined how job contents evolve in the information age despite the fact that new technologies typically substitute for specific job tasks, s
hifting job skills rather than eliminating whole jobs. Here we explore the process and consequences of changes in occupational skill contents and characterize occupations subject to the most re-skilling pressure. Recent research suggests that high-skilled STEM and technology-intensive business occupations have experienced the highest rates of skill content change. Using a dataset covering the near universe of U.S. online job postings between 2010 and 2018, we find that when the number and similarity of skills within a job are taken into account, the re-skilling pressure is much higher for workers in low complexity, low education and low compensation occupations. We use high-dimensional embeddings of skills estimated across all jobs to precisely assess skill similarity, and characterize occupational skill transformations, demonstrating that skills requiring machine-operation and interface rise sharply in importance in the past decade, much more than human interface skills in low and mid-education occupations. We establish that large organizations buffer jobs from skill instability and obsolescence, especially low-skilled jobs with unstable skill requirements. Finally, the gap in re-skilling pressure between low/mid-education and high-education occupations is smaller in large organizations, suggesting that by controlling the surrounding skill environment, such organizations reduce the rate of required re-skilling and sustain short-term productivity for those occupations.
A question answering (QA) system is a type of conversational AI that generates natural language answers to questions posed by human users. QA systems often form the backbone of interactive dialogue systems, and have been studied extensively for a wid
e variety of tasks ranging from restaurant recommendations to medical diagnostics. Dramatic progress has been made in recent years, especially from the use of encoder-decoder neural architectures trained with big data input. In this paper, we take initial steps to bringing state-of-the-art neural QA technologies to Software Engineering applications by designing a context-based QA system for basic questions about subroutines. We curate a training dataset of 10.9 million question/context/answer tuples based on rules we extract from recent empirical studies. Then, we train a custom neural QA model with this dataset and evaluate the model in a study with professional programmers. We demonstrate the strengths and weaknesses of the system, and lay the groundwork for its use in eventual dialogue systems for software engineering.
Recognizing named entities (NEs) is commonly conducted as a classification problem that predicts a class tag for an NE candidate in a sentence. In shallow structures, categorized features are weighted to support the prediction. Recent developments in
neural networks have adopted deep structures that map categorized features into continuous representations. This approach unfolds a dense space saturated with high-order abstract semantic information, where the prediction is based on distributed feature representations. In this paper, the regression operation is introduced to locate NEs in a sentence. In this approach, a deep network is first designed to transform an input sentence into recurrent feature maps. Bounding boxes are generated from the feature maps, where a box is an abstract representation of an NE candidate. In addition to the class tag, each bounding box has two parameters denoting the start position and the length of an NE candidate. In the training process, the location offset between a bounding box and a true NE are learned to minimize the location loss. Based on this motivation, a multiobjective learning framework is designed to simultaneously locate entities and predict the class probability. By sharing parameters for locating and predicting, the framework can take full advantage of annotated data and enable more potent nonlinear function approximators to enhance model discriminability. Experiments demonstrate state-of-the-art performance for nested named entitiesfootnote{Our codes will be available at: url{https://github.com/wuyuefei3/BR}}.
