Federated learning (FL) is a privacy-preserving machine learning paradigm that enables collaborative training among geographically distributed and heterogeneous users without gathering their data. Extending FL beyond the conventional supervised learn
ing paradigm, federated Reinforcement Learning (RL) was proposed to handle sequential decision-making problems for various privacy-sensitive applications such as autonomous driving. However, the existing federated RL algorithms directly combine model-free RL with FL, and thus generally have high sample complexity and lack theoretical guarantees. To address the above challenges, we propose a new federated RL algorithm that incorporates model-based RL and ensemble knowledge distillation into FL. Specifically, we utilise FL and knowledge distillation to create an ensemble of dynamics models from clients, and then train the policy by solely using the ensemble model without interacting with the real environment. Furthermore, we theoretically prove that the monotonic improvement of the proposed algorithm is guaranteed. Extensive experimental results demonstrate that our algorithm obtains significantly higher sample efficiency compared to federated model-free RL algorithms in the challenging continuous control benchmark environments. The results also show the impact of non-IID client data and local update steps on the performance of federated RL, validating the insights obtained from our theoretical analysis.
In this paper we introduce the so-called truncated very-well-poised $_6psi_6$ series and set up an explicit recurrence relation for it by means of the classical Abel lemma on summation by parts. This new recurrence relation implies an elementary proo
f of Baileys well-known $_6psi_6$ summation formula.
Temporal relational data, perhaps the most commonly used data type in industrial machine learning applications, needs labor-intensive feature engineering and data analyzing for giving precise model predictions. An automatic machine learning framework
is needed to ease the manual efforts in fine-tuning the models so that the experts can focus more on other problems that really need humans engagement such as problem definition, deployment, and business services. However, there are three main challenges for building automatic solutions for temporal relational data: 1) how to effectively and automatically mining useful information from the multiple tables and the relations from them? 2) how to be self-adjustable to control the time and memory consumption within a certain budget? and 3) how to give generic solutions to a wide range of tasks? In this work, we propose our solution that successfully addresses the above issues in an end-to-end automatic way. The proposed framework, AutoSmart, is the winning solution to the KDD Cup 2019 of the AutoML Track, which is one of the largest AutoML competition to date (860 teams with around 4,955 submissions). The framework includes automatic data processing, table merging, feature engineering, and model tuning, with a time&memory controller for efficiently and automatically formulating the models. The proposed framework outperforms the baseline solution significantly on several datasets in various domains.
This paper introduces a novel motion planning algorithm, incrementally stochastic and accelerated gradient information mixed optimization (iSAGO), for robotic manipulators in a narrow workspace. Primarily, we propose the overall scheme of iSAGO integ
rating the accelerated and stochastic gradient information for efficient descent in the penalty method. In the stochastic part, we generate the adaptive stochastic moment via the random selection of collision checkboxes, interval time-series, and penalty factor based on Adam to solve the body-obstacle stuck case. Due to the slow convergence of STOMA, we integrate the accelerated gradient and stimulate the descent rate in a Lipschitz constant reestimation framework. Moreover, we introduce the Bayesian tree inference (BTI) method, transforming the whole trajectory optimization (SAGO) into an incremental sub-trajectory optimization (iSAGO) to improve the computational efficiency and success rate. Finally, we demonstrate the key coefficient tuning, benchmark iSAGO against other planners (CHOMP, GPMP2, TrajOpt, STOMP, and RRT-Connect), and implement iSAGO on AUBO-i5 in a storage shelf. The result shows the highest success rate and moderate solving efficiency of iSAGO.
Photon pairs produced by the pulse-pumped nonlinear parametric processes have been a workhorse of quantum information science. Engineering the spectral property of the photon pairs is crucial in practical applications. In this article, we demonstrate
a programmable photon pair source by exploiting a two-stage nonlinear interferometer with a phase-control device. The phase-control device introduces phase shifts by a programmable phase function that can be arbitrarily defined. With a properly designed phase function, the output spectrum of the source can be freely customized and changed without replacing any hardware component in the system. In addition to demonstrating the generation of photon pairs with factorable, positively-correlated, and negatively-correlated spectra, respectively, we show that the output of the source can be tailored into multi-channel spectrally factorable photon pairs without sacrificing efficiency. Such a source, having the ability to modify the spectrum of the photon pairs at will according to the chosen application, is a powerful tool for quantum information science.
Glasses possess more low-frequency vibrational modes than predicted by Debye theory. These excess modes are crucial for the understanding the low temperature thermal and mechanical properties of glasses, which differ from those of crystalline solids.
Recent simulational studies suggest that the density of the excess modes scales with their frequency $omega$ as $omega^4$ in two and higher dimensions. Here, we present extensive numerical studies of two-dimensional model glass formers over a large range of glass stabilities. We find that the density of the excess modes follows $D_text{exc}(omega)sim omega^2 $ up to around the boson peak, regardless of the glass stability. The stability dependence of the overall scale of $D_text{exc}(omega)$ correlates with the stability dependence of low-frequency sound attenuation. However, we also find that in small systems, where the first sound mode is pushed to higher frequencies, at frequencies below the first sound mode there are excess modes with a system size independent density of states that scales as $omega^3$.
Detection in large-scale scenes is a challenging problem due to small objects and extreme scale variation. It is essential to focus on the image regions of small objects. In this paper, we propose a novel Adaptive Zoom (AdaZoom) network as a selectiv
e magnifier with flexible shape and focal length to adaptively zoom the focus regions for object detection. Based on policy gradient, we construct a reinforcement learning framework for focus region generation, with the reward formulated by object distributions. The scales and aspect ratios of the generated regions are adaptive to the scales and distribution of objects inside. We apply variable magnification according to the scale of the region for adaptive multi-scale detection. We further propose collaborative training to complementarily promote the performance of AdaZoom and the detection network. To validate the effectiveness, we conduct extensive experiments on VisDrone2019, UAVDT, and DOTA datasets. The experiments show AdaZoom brings a consistent and significant improvement over different detection networks, achieving state-of-the-art performance on these datasets, especially outperforming the existing methods by AP of 4.64% on Vis-Drone2019.
We report here a linear fluoride-bridged Double-Decker Dysprosium metallacrown with high-performance SMM. The successful introduction of stronger magnetic exchange-coupling in the axial direction, which is collinear with the Ising-type magnetic aniso
tropy axis of dysprosium ions, plays a pivotal role in improving the SMM properties of the double-decker Dysprosium metallacrown.
Non-intrusive load monitoring (NILM) is essential for understanding customers power consumption patterns and may find wide applications like carbon emission reduction and energy conservation. The training of NILM models requires massive load data con
taining different types of appliances. However, inadequate load data and the risk of power consumer privacy breaches may be encountered by local data owners during the NILM model training. To prevent such potential risks, a novel NILM method named Fed-NILM which is based on Federated Learning (FL) is proposed in this paper. In Fed-NILM, local model parameters instead of local load data are shared among multiple data owners. The global model is obtained by weighted averaging the parameters. Experiments based on two measured load datasets are conducted to explore the generalization ability of Fed-NILM. Besides, a comparison of Fed-NILM with locally-trained NILMs and the centrally-trained NILM is conducted. The experimental results show that Fed-NILM has superior performance in scalability and convergence. Fed-NILM outperforms locally-trained NILMs operated by local data owners and approximates the centrally-trained NILM which is trained on the entire load dataset without privacy protection. The proposed Fed-NILM significantly improves the co-modeling capabilities of local data owners while protecting power consumers privacy.
Although the combination of the orthogonal time frequency space (OTFS) modulation and the massive multiple-input multiple-output (MIMO) technology can make communication systems perform better in high-mobility scenarios, there are still many challeng
es in downlink channel estimation owing to inaccurate modeling and high pilot overhead in practical systems. In this paper, we propose a channel state information (CSI) acquisition scheme for downlink massive MIMO-OTFS in presence of the fractional Doppler, including deterministic pilot design and channel estimation algorithm. First, we analyze the input-output relationship of the single-input single-output (SISO) OTFS based on the orthogonal frequency division multiplexing (OFDM) modem and extend it to massive MIMO-OTFS. Moreover, we formulate an accurate model for the practical system in which the fractional Doppler is considered and the influence of subpaths is revealed. A deterministic pilot design is then proposed based on the model and the structure of the pilot matrix to reduce pilot overhead and save memory consumption. Since channel geometry changes very slowly relative to the communication timescale, we put forward a modified sensing matrix based channel estimation (MSMCE) algorithm to acquire the downlink CSI. Simulation results demonstrate that the proposed downlink CSI acquisition scheme has significant advantages over traditional algorithms.
