One-dimensional electron systems (1DESs) exhibit properties that are fundamentally different from higher-dimensional systems. For example, electron-electron interactions in 1DESs have been predicted to induce Tomonaga-Luttinger liquid behavior. Natur
ally-occurring grain boundaries in single-layer semiconducting transition metal dichalcogenides provide 1D conducting channels that have been proposed to host Tomonaga-Luttinger liquids, but charge density wave physics has also been suggested to explain their behavior. Clear identification of the electronic ground state of this system has been hampered by an inability to electrostatically gate such boundaries and thereby tune their charge carrier concentration. Here we present a scanning tunneling microscopy/spectroscopy study of gate-tunable mirror twin boundaries (MTBs) in single-layer 1H-MoSe$_2$ devices. Gating here enables STM spectroscopy to be performed for different MTB electron densities, thus allowing precise characterization of electron-electron interaction effects. Visualization of MTB electronic structure under these conditions allows unambiguous identification of collective density wave excitations having two distinct velocities, in quantitative agreement with the spin-charge separation predicted by finite-length Tomonaga-Luttinger-liquid theory.
Sequential deep learning models such as RNN, causal CNN and attention mechanism do not readily consume continuous-time information. Discretizing the temporal data, as we show, causes inconsistency even for simple continuous-time processes. Current ap
proaches often handle time in a heuristic manner to be consistent with the existing deep learning architectures and implementations. In this paper, we provide a principled way to characterize continuous-time systems using deep learning tools. Notably, the proposed approach applies to all the major deep learning architectures and requires little modifications to the implementation. The critical insight is to represent the continuous-time system by composing neural networks with a temporal kernel, where we gain our intuition from the recent advancements in understanding deep learning with Gaussian process and neural tangent kernel. To represent the temporal kernel, we introduce the random feature approach and convert the kernel learning problem to spectral density estimation under reparameterization. We further prove the convergence and consistency results even when the temporal kernel is non-stationary, and the spectral density is misspecified. The simulations and real-data experiments demonstrate the empirical effectiveness of our temporal kernel approach in a broad range of settings.
The recent paper by Byrd & Lipton (2019), based on empirical observations, raises a major concern on the impact of importance weighting for the over-parameterized deep learning models. They observe that as long as the model can separate the training
data, the impact of importance weighting diminishes as the training proceeds. Nevertheless, there lacks a rigorous characterization of this phenomenon. In this paper, we provide formal characterizations and theoretical justifications on the role of importance weighting with respect to the implicit bias of gradient descent and margin-based learning theory. We reveal both the optimization dynamics and generalization performance under deep learning models. Our work not only explains the various novel phenomenons observed for importance weighting in deep learning, but also extends to the studies where the weights are being optimized as part of the model, which applies to a number of topics under active research.
Product embeddings have been heavily investigated in the past few years, serving as the cornerstone for a broad range of machine learning applications in e-commerce. Despite the empirical success of product embeddings, little is known on how and why
they work from the theoretical standpoint. Analogous results from the natural language processing (NLP) often rely on domain-specific properties that are not transferable to the e-commerce setting, and the downstream tasks often focus on different aspects of the embeddings. We take an e-commerce-oriented view of the product embeddings and reveal a complete theoretical view from both the representation learning and the learning theory perspective. We prove that product embeddings trained by the widely-adopted skip-gram negative sampling algorithm and its variants are sufficient dimension reduction regarding a critical product relatedness measure. The generalization performance in the downstream machine learning task is controlled by the alignment between the embeddings and the product relatedness measure. Following the theoretical discoveries, we conduct exploratory experiments that supports our theoretical insights for the product embeddings.
The feedback data of recommender systems are often subject to what was exposed to the users; however, most learning and evaluation methods do not account for the underlying exposure mechanism. We first show in theory that applying supervised learning
to detect user preferences may end up with inconsistent results in the absence of exposure information. The counterfactual propensity-weighting approach from causal inference can account for the exposure mechanism; nevertheless, the partial-observation nature of the feedback data can cause identifiability issues. We propose a principled solution by introducing a minimax empirical risk formulation. We show that the relaxation of the dual problem can be converted to an adversarial game between two recommendation models, where the opponent of the candidate model characterizes the underlying exposure mechanism. We provide learning bounds and conduct extensive simulation studies to illustrate and justify the proposed approach over a broad range of recommendation settings, which shed insights on the various benefits of the proposed approach.
Two-dimensional triangular-lattice antiferromagnets are predicted under some conditions to exhibit a quantum spin liquid ground state whose low-energy behavior is described by a spinon Fermi surface. Directly imaging the resulting spinons, however, i
s difficult due to their fractional, chargeless nature. Here we use scanning tunneling spectroscopy to image spinon density modulations arising from a spinon Fermi surface instability in single-layer 1T-TaSe$_2$, a two-dimensional Mott insulator. We first demonstrate the existence of localized spins arranged on a triangular lattice in single-layer 1T-TaSe$_2$ by contacting it to a metallic 1H-TaSe$_2$ layer and measuring the Kondo effect. Subsequent spectroscopic imaging of isolated, single-layer 1T-TaSe$_2$ reveals long-wavelength modulations at Hubbard band energies that reflect spinon density modulations. This allows direct experimental measurement of the spinon Fermi wavevector, in good agreement with theoretical predictions for a 2D quantum spin liquid. These results establish single-layer 1T-TaSe$_2$ as a new platform for studying novel two-dimensional quantum-spin-liquid phenomena.
Inductive representation learning on temporal graphs is an important step toward salable machine learning on real-world dynamic networks. The evolving nature of temporal dynamic graphs requires handling new nodes as well as capturing temporal pattern
s. The node embeddings, which are now functions of time, should represent both the static node features and the evolving topological structures. Moreover, node and topological features can be temporal as well, whose patterns the node embeddings should also capture. We propose the temporal graph attention (TGAT) layer to efficiently aggregate temporal-topological neighborhood features as well as to learn the time-feature interactions. For TGAT, we use the self-attention mechanism as building block and develop a novel functional time encoding technique based on the classical Bochners theorem from harmonic analysis. By stacking TGAT layers, the network recognizes the node embeddings as functions of time and is able to inductively infer embeddings for both new and observed nodes as the graph evolves. The proposed approach handles both node classification and link prediction task, and can be naturally extended to include the temporal edge features. We evaluate our method with transductive and inductive tasks under temporal settings with two benchmark and one industrial dataset. Our TGAT model compares favorably to state-of-the-art baselines as well as the previous temporal graph embedding approaches.
Sequential modelling with self-attention has achieved cutting edge performances in natural language processing. With advantages in model flexibility, computation complexity and interpretability, self-attention is gradually becoming a key component in
event sequence models. However, like most other sequence models, self-attention does not account for the time span between events and thus captures sequential signals rather than temporal patterns. Without relying on recurrent network structures, self-attention recognizes event orderings via positional encoding. To bridge the gap between modelling time-independent and time-dependent event sequence, we introduce a functional feature map that embeds time span into high-dimensional spaces. By constructing the associated translation-invariant time kernel function, we reveal the functional forms of the feature map under classic functional function analysis results, namely Bochners Theorem and Mercers Theorem. We propose several models to learn the functional time representation and the interactions with event representation. These methods are evaluated on real-world datasets under various continuous-time event sequence prediction tasks. The experiments reveal that the proposed methods compare favorably to baseline models while also capturing useful time-event interactions.
In this paper, we propose a new product knowledge graph (PKG) embedding approach for learning the intrinsic product relations as product knowledge for e-commerce. We define the key entities and summarize the pivotal product relations that are critica
l for general e-commerce applications including marketing, advertisement, search ranking and recommendation. We first provide a comprehensive comparison between PKG and ordinary knowledge graph (KG) and then illustrate why KG embedding methods are not suitable for PKG learning. We construct a self-attention-enhanced distributed representation learning model for learning PKG embeddings from raw customer activity data in an end-to-end fashion. We design an effective multi-task learning schema to fully leverage the multi-modal e-commerce data. The Poincare embedding is also employed to handle complex entity structures. We use a real-world dataset from grocery.walmart.com to evaluate the performances on knowledge completion, search ranking and recommendation. The proposed approach compares favourably to baselines in knowledge completion and downstream tasks.
A major obstacle in understanding the mechanism of Cooper pairing in the cuprates is the existence of various intertwined orders associated with spin, charge, and Cooper pairs. Of particular importance is the ubiquitous charge order features that hav
e been observed in a variety of cuprates, especially in the underdoped regime of the phase diagram. To explain the origin of the charge order and its implication to the superconducting phase, many theoretical models have been proposed, such as charge stripes, electronic nematicity, and Fermi surface instability. A highly appealing physical picture is the so-called pair density wave (PDW), a periodic modulation of Cooper paring in space, which may also induce a charge order. To elucidate the existence and nature of the PDW order, here we use scanning tunneling microscopy (STM) to investigate a severely underdoped Bi2Sr2CaCu2O8+{delta}, in which superconductivity just emerges on top of a pronounced checkerboard charge order. By analyzing the spatial distribution of the spectral features characteristic of superconductivity, we observe a periodic modulation of both the superconducting coherence peak and gap depth, demonstrating the existence of a density wave order of Cooper pairing. The PDW order has the same spatial periodicity as the charge order, and the amplitudes of the two orders exhibit clear positive correlation. These results shed important new lights on the origin of and interplay between the charge order and Cooper pairing modulation in the cuprates.
