We consider the problem of offline reinforcement learning with model-based control, whose goal is to learn a dynamics model from the experience replay and obtain a pessimism-oriented agent under the learned model. Current model-based constraint inclu
des explicit uncertainty penalty and implicit conservative regularization that pushes Q-values of out-of-distribution state-action pairs down and the in-distribution up. While the uncertainty estimation, on which the former relies on, can be loosely calibrated for complex dynamics, the latter performs slightly better. To extend the basic idea of regularization without uncertainty quantification, we propose distributionally robust offline model-based policy optimization (DROMO), which leverages the ideas in distributionally robust optimization to penalize a broader range of out-of-distribution state-action pairs beyond the standard empirical out-of-distribution Q-value minimization. We theoretically show that our method optimizes a lower bound on the ground-truth policy evaluation, and it can be incorporated into any existing policy gradient algorithms. We also analyze the theoretical properties of DROMOs linear and non-linear instantiations.
In large-scale image retrieval, many indexing methods have been proposed to narrow down the searching scope of retrieval. The features extracted from images usually are of high dimensions or unfixed sizes due to the existence of key points. Most of e
xisting index structures suffer from the dimension curse, the unfixed feature size and/or the loss of semantic similarity. In this paper a new classification-based indexing structure, called Semantic Indexing Structure (SIS), is proposed, in which we utilize the semantic categories rather than clustering centers to create database partitions, such that the proposed index SIS can be combined with feature extractors without the restriction of dimensions. Besides, it is observed that the size of each semantic partition is positively correlated with the semantic distribution of database. Along this way, we found that when the partition number is normalized to five, the proposed algorithm performed very well in all the tests. Compared with state-of-the-art models, SIS achieves outstanding performance.
We study the tagging of Higgs exotic decay signals using different types of deep neural networks (DNNs), focusing on the $W^pm h$ associated production channel followed by Higgs decaying into $n$ $b$-quarks with $n=4$, 6 and 8. All the Higgs decay pr
oducts are collected into a fat-jet, to which we apply further selection using the DNNs. Three kinds of DNNs are considered, namely convolutional neural network (CNN), recursive neural network (RecNN) and particle flow network (PFN). The PFN can achieve the best performance because its structure allows enfolding more information in addition to the four-momentums of the jet constituents, such as particle ID and tracks parameters. Using the PFN as an example, we verify that it can serve as an efficient tagger even though it is trained on a different event topology with different $b$-multiplicity from the actual signal. The projected sensitivity to the branching ratio of Higgs decaying into $n$ $b$-quarks at the HL-LHC are 10%, 3% and 1%, for $n=4$, 6 and 8, respectively.
The measurement of the arrival time of a particle, such as a lepton, a photon, or a pion, reaching the detector provides valuable information. A similar measurement for a hadronic final state, however, is much more challenging as one has to extract t
he relevant information from a collection of particles. In this paper, we explore various possibilities in defining the time of a jet through the measurable arrival times of the jet constituents. We find that a definition of jet time based on a transverse momentum weighted sum of the times of the constituents has the best performance. For prompt jets, the performance depends on the jet trajectory. For delayed jets, the performance depends on the trajectory of the jet, the trajectory of the mother particle, and the location of the displaced vertex. Compared to the next-best-performing jet time definition, the transverse momentum weighted sum has roughly a factor of ten times better jet time resolution. We give a detailed discussion of the relevant effects and characterize the full geometrical dependence of the performance. These results highlight the critical importance of using a proper definition of jet time with its corresponding detector-dependent calibration and the exciting possibility of deepening our understanding of jets in the time domain.
The switching behavior of antiferroelectric domain structures under the applied electric field is not fully understood. In this work, by using the phase field simulation, we have studied the polarization switching property of antiferroelectric domain
s. Our results indicate that the ferroelectric domains nucleate preferably at the boundaries of the antiferroelectric domains, and antiferroelectrics with larger initial domain sizes possess a higher coercive electric field as demonstrated by hysteresis loops. Moreover, we introduced charge defects into the sample and numerically investigated their influence. It is also shown that charge defects can induce local ferroelectric domains, which could suppress the saturation polarization and narrow the enclosed area of the hysteresis loop. Our results give insights into understanding antiferroelectric phase transformation and optimizing the energy storage property in experiments.
We present an end-to-end, model-based deep reinforcement learning agent which dynamically attends to relevant parts of its state, in order to plan and to generalize better out-of-distribution. The agents architecture uses a set representation and a b
ottleneck mechanism, forcing the number of entities to which the agent attends at each planning step to be small. In experiments with customized MiniGrid environments with different dynamics, we observe that the design allows agents to learn to plan effectively, by attending to the relevant objects, leading to better out-of-distribution generalization.
Knowledge Graph has been proven effective in modeling structured information and conceptual knowledge, especially in the medical domain. However, the lack of high-quality annotated corpora remains a crucial problem for advancing the research and appl
ications on this task. In order to accelerate the research for domain-specific knowledge graphs in the medical domain, we introduce DiaKG, a high-quality Chinese dataset for Diabetes knowledge graph, which contains 22,050 entities and 6,890 relations in total. We implement recent typical methods for Named Entity Recognition and Relation Extraction as a benchmark to evaluate the proposed dataset thoroughly. Empirical results show that the DiaKG is challenging for most existing methods and further analysis is conducted to discuss future research direction for improvements. We hope the release of this dataset can assist the construction of diabetes knowledge graphs and facilitate AI-based applications.
A community reveals the features and connections of its members that are different from those in other communities in a network. Detecting communities is of great significance in network analysis. Despite the classical spectral clustering and statist
ical inference methods, we notice a significant development of deep learning techniques for community detection in recent years with their advantages in handling high dimensional network data. Hence, a comprehensive overview of community detections latest progress through deep learning is timely to both academics and practitioners. This survey devises and proposes a new taxonomy covering different categories of the state-of-the-art methods, including deep learning-based models upon deep neural networks, deep nonnegative matrix factorization and deep sparse filtering. The main category, i.e., deep neural networks, is further divided into convolutional networks, graph attention networks, generative adversarial networks and autoencoders. The survey also summarizes the popular benchmark data sets, model evaluation metrics, and open-source implementations to address experimentation settings. We then discuss the practical applications of community detection in various domains and point to implementation scenarios. Finally, we outline future directions by suggesting challenging topics in this fast-growing deep learning field.
In this paper, we present an attention-guided deformable convolutional network for hand-held multi-frame high dynamic range (HDR) imaging, namely ADNet. This problem comprises two intractable challenges of how to handle saturation and noise properly
and how to tackle misalignments caused by object motion or camera jittering. To address the former, we adopt a spatial attention module to adaptively select the most appropriate regions of various exposure low dynamic range (LDR) images for fusion. For the latter one, we propose to align the gamma-corrected images in the feature-level with a Pyramid, Cascading and Deformable (PCD) alignment module. The proposed ADNet shows state-of-the-art performance compared with previous methods, achieving a PSNR-$l$ of 39.4471 and a PSNR-$mu$ of 37.6359 in NTIRE 2021 Multi-Frame HDR Challenge.
Electroweak symmetry non-restoration up to high temperatures well above the electroweak scale offers new alternatives for baryogenesis. We propose a new approach for electroweak symmetry non-restoration via an inert Higgs sector that couples to the S
tandard Model Higgs as well as an extended scalar singlet sector. We implement renormalization group improvements and thermal resummation, necessary to evaluate the effective potential spanning over a broad range of energy scales and temperatures. We present examples of benchmark scenarios that allow for electroweak symmetry non-restoration all the way up to hundreds of TeV temperatures, and also feature suppressed sphaleron washout factors down to the electroweak scale. Our method for transmitting the Standard Model broken electroweak symmetry to an inert Higgs sector has several intriguing implications for (electroweak) baryogenesis, early universe thermal histories, and can be scrutinized through Higgs physics phenomenology and electroweak precision measurements at the HL-LHC.
