Adaptive optimization methods have been widely used in deep learning. They scale the learning rates adaptively according to the past gradient, which has been shown to be effective to accelerate the convergence. However, they suffer from poor generali
zation performance compared with SGD. Recent studies point that smoothing exponential gradient noise leads to generalization degeneration phenomenon. Inspired by this, we propose AdaL, with a transformation on the original gradient. AdaL accelerates the convergence by amplifying the gradient in the early stage, as well as dampens the oscillation and stabilizes the optimization by shrinking the gradient later. Such modification alleviates the smoothness of gradient noise, which produces better generalization performance. We have theoretically proved the convergence of AdaL and demonstrated its effectiveness on several benchmarks.
Multivariate time series prediction has attracted a lot of attention because of its wide applications such as intelligence transportation, AIOps. Generative models have achieved impressive results in time series modeling because they can model data d
istribution and take noise into consideration. However, many existing works can not be widely used because of the constraints of functional form of generative models or the sensitivity to hyperparameters. In this paper, we propose ScoreGrad, a multivariate probabilistic time series forecasting framework based on continuous energy-based generative models. ScoreGrad is composed of time series feature extraction module and conditional stochastic differential equation based score matching module. The prediction can be achieved by iteratively solving reverse-time SDE. To the best of our knowledge, ScoreGrad is the first continuous energy based generative model used for time series forecasting. Furthermore, ScoreGrad achieves state-of-the-art results on six real-world datasets. The impact of hyperparameters and sampler types on the performance are also explored. Code is available at https://github.com/yantijin/ScoreGradPred.
The Anderson localization of thermal phonons has been shown only in few nano-structures with strong random disorder by the exponential decay of transmission to zero and a thermal conductivity maximum when increasing system length. In this work, we pr
esent a path to demonstrate the phonon localization with distinctive features in graded superlattices with short-range order and long-range disorder. A thermal conductivity minimum with system length appears due to the exponential decay of transmission to a non-zero constant, which is a feature of partial phonon localization caused by the moderate disorder. We provide clear evidence of localization through the combined analysis of the participation ratio, transmission, and real-space phonon number density distribution based on our quantum transport simulation. The present work would promote heat conduction engineering by localization via the wave nature of phonons.
The understanding and modeling of inelastic scattering of thermal phonons at a solid/solid interface remain an open question. We present a fully quantum theoretical scheme to quantify the effect of anharmonic phonon-phonon scattering at an interface
via non-equilibrium Greens function (NEGF) formalism. Based on the real-space scattering rate matrix, a decomposition of the interfacial spectral energy exchange is made into contributions from local and non-local anharmonic interactions, of which the former is shown to be predominant for high-frequency phonons whereas both are important for low-frequency phonons. The anharmonic decay of interfacial phonon modes is revealed to play a crucial role in bridging the bulk modes across the interface. The overall quantitative contribution of anharmonicity to thermal boundary conductance is found to be moderate. The present work promotes a deeper understanding of heat transport at the interface and an intuitive interpretation of anharmonic phonon NEGF formalism.
Radio access network (RAN) virtualization is gaining more and more ground and expected to re-architect the next-generation cellular networks. Existing RAN virtualization studies and solutions have mostly focused on sharing communication capacity and
tend to require the use of the same PHY and MAC layers across network slices. This approach has not considered the scenarios where different slices require different PHY and MAC layers, for instance, for radically different services and for whole-stack research in wireless living labs where novel PHY and MAC layers need to be deployed concurrently with existing ones on the same physical infrastructure. To enable whole-stack slicing where different PHY and MAC layers may be deployed in different slices, we develop PV-RAN, the first open-source virtual RAN platform that enables the sharing of the same SDR physical resources across multiple slices. Through API Remoting, PV-RAN enables running paravirtualized instances of OpenAirInterface (OAI) at different slices without requiring modifying OAI source code. PV-RAN effectively leverages the inter-domain communication mechanisms of Xen to transport time-sensitive I/Q samples via shared memory, making the virtualization overhead in communication almost negligible. We conduct detailed performance benchmarking of PV-RAN and demonstrate its low overhead and high efficiency. We also integrate PV-RAN with the CyNet wireless living lab for smart agriculture and transportation.
Graph convolutional networks (GCNs) have achieved promising performance on various graph-based tasks. However they suffer from over-smoothing when stacking more layers. In this paper, we present a quantitative study on this observation and develop no
vel insights towards the deeper GCN. First, we interpret the current graph convolutional operations from an optimization perspective and argue that over-smoothing is mainly caused by the naive first-order approximation of the solution to the optimization problem. Subsequently, we introduce two metrics to measure the over-smoothing on node-level tasks. Specifically, we calculate the fraction of the pairwise distance between connected and disconnected nodes to the overall distance respectively. Based on our theoretical and empirical analysis, we establish a universal theoretical framework of GCN from an optimization perspective and derive a novel convolutional kernel named GCN+ which has lower parameter amount while relieving the over-smoothing inherently. Extensive experiments on real-world datasets demonstrate the superior performance of GCN+ over state-of-the-art baseline methods on the node classification tasks.
The coherent quantum effect becomes increasingly important in the heat dissipation bottleneck of semiconductor nanoelectronics with the characteristic size shrinking down to few nano-meters scale nowadays. However, the quantum mechanical model remain
s elusive for anharmonic phonon-phonon scattering in extremely small nanostructures with broken translational symmetry. It is a long-term challenging task to correctly simulate quantum heat transport including anharmonic scattering at a scale relevant to practical applications. In this article, we present a clarified theoretical formulation of anharmonic phonon non-equilibrium Green function (NEGF) formalism for both 1D and 3D nanostructures, through a diagrammatic perturbation expansion and an introduction of Fourier representation to both harmonic and anharmonic terms. A parallelized computational framework with first-principle force constants input is developed for large-scale quantum heat transport simulation. Some crucial approximations in numerical implementation are investigated to ensure the balance between numerical accuracy and efficiency. A quantitative validation is demonstrated for the anharmonic phonon NEGF formalism and computational framework by modeling cross-plane heat transport through silicon thin film. The phonon-phonon scattering is shown to be appreciable and to introduce about 20% reduction of thermal conductivity at room temperature even for a film thickness around 10 nm. The present methodology provides a robust platform for the device quantum thermal modeling, as well as the study on the transition from coherent to incoherent heat transport in nano-phononic crystals. This work thus paves the way to understand and to manipulate heat conduction via the wave nature of phonons.
Self-synchronization is a ubiquitous phenomenon in nature, in which oscillators are collectively locked in frequency and phase through mutual interactions. While self-synchronization requires the forced excitation of at least one of the oscillators,
we demonstrate that this mechanism spontaneously appears due to the activation from thermal fluctuations. By performing molecular dynamic simulations, we demonstrate the self-synchronization of thermal phonons in a platform supporting doped silicon resonators. We find that thermal phonons are spontaneously converging to the same frequency and phase. In addition, the dependencies to intrinsic frequency difference and coupling strength agree well with the Kuramoto model predictions. More interestingly, we find that a balance between energy dissipation resulting from phonon-phonon scattering and potential energy between oscillators is required to maintain synchronization. Finally, a wavelet transform approach corroborates the generation of coherent thermal phonons in the collective state of oscillators. Our study provides a new perspective on self-synchronization and on the relationship between fluctuations and coherence.
Our direct atomic simulations reveal that a thermally activated phonon mode involves a large population of elastic wavepackets. These excitations are characterized by a wide distribution of lifetimes and coherence times expressing particle- and wave-
like natures. In agreement with direct simulations, our theoretical derivation yields a generalized law for the decay of the phonon number taking into account coherent effects. Before the conventional exponential decay due to phonon-phonon scattering, this law introduces a delay proportional to the square of the coherence time. This additional regime leads to a moderate increase in relaxation times and thermal conductivity. This work opens new horizons in the understanding of the origin and the treatment of thermal phonons.
We present the systematic de Haas-van Alphen (dHvA) quantum oscillations studies on the recently discovered topological Dirac semimetal pyrite PtBi2 single crystals. Remarkable dHvA oscillations were observed at field as low as 1.5 T. From the analys
es of dHvA oscillations, we have extracted high quantum mobility, light effective mass and phase shift factor for Dirac fermions in pyrite PtBi2. From the angular dependence of dHvA oscillations, we have mapped out the topology of the Fermi surface and identified additional oscillation frequencies which were not probed by SdH oscillations.
