Improving Long-Tailed Classification from Instance Level

Data in the real world tends to exhibit a long-tailed label distribution, which poses great challenges for neural networks in classification. Existing methods tackle this problem mainly from the coarse-grained class level, ignoring the difference among instances, e.g., hard samples vs. easy samples. In this paper, we revisit the long-tailed problem from the instance level and propose two instance-level components to improve long-tailed classification. The first one is an Adaptive Logit Adjustment (ALA) loss, which applies an adaptive adjusting term to the logit. Different from the adjusting terms in existing methods that are class-dependent and only focus on tail classes, we carefully design an instance-specific term and add it on the class-dependent term to make the network pay more attention to not only tailed class, but more importantly hard samples. The second one is a Mixture-of-Experts (MoE) network, which contains a multi-expert module and an instance-aware routing module. The routing module is designed to dynamically integrate the results of multiple experts according to each input instance, and is trained jointly with the experts network in an end-to-end manner.Extensive experiment results show that our method outperforms the state-of-the-art methods by 1% to 5% on common long-tailed benchmarks including ImageNet-LT and iNaturalist.

Sparse Array of Sub-surface Aided Anti-blockage mmWave Communication Systems

Recently, reconfigurable intelligent surfaces (RISs) have drawn intensive attention to enhance the coverage of millimeter wave (mmWave) communication systems. However, existing works mainly consider the RIS as a whole uniform plane, which may be unrealistic to be installed on the facade of buildings when the RIS is extreme large. To address this problem, in this paper, we propose a sparse array of sub-surface (SAoS) architecture for RIS, which contains several rectangle shaped sub-surfaces termed as RIS tiles that can be sparsely deployed. An approximated ergodic spectral efficiency of the SAoS aided system is derived and the performance impact of the SAoS design is evaluated. Based on the approximated ergodic spectral efficiency, we obtain an optimal reflection coefficient design for each RIS tile. Analytical results show that the received signal-to-noise ratios can grow quadratically and linearly to the number of RIS elements under strong and weak LoS scenarios, respectively. Furthermore, we consider the visible region (VR) phenomenon in the SAoS aided mmWave system and find that the optimal distance between RIS tiles is supposed to yield a total SAoS VR nearly covering the whole blind coverage area. The numerical results verify the tightness of the approximated ergodic spectral efficiency and demonstrate the great system performance.

Adaptive Bit Partitioning for Reconfigurable Intelligent Surface Assisted FDD Systems with Limited Feedback

In frequency division duplexing systems, the base station (BS) acquires downlink channel state information (CSI) via channel feedback, which has not been adequately investigated in the presence of RIS. In this study, we examine the limited channel feedback scheme by proposing a novel cascaded codebook and an adaptive bit partitioning strategy. The RIS segments the channel between the BS and mobile station into two sub-channels, each with line-of-sight (LoS) and non-LoS (NLoS) paths. To quantize the path gains, the cascaded codebook is proposed to be synthesized by two sub-codebooks whose codeword is cascaded by LoS and NLoS components. This enables the proposed cascaded codebook to cater the different distributions of LoS and NLoS path gains by flexibly using different feedback bits to design the codeword structure. On the basis of the proposed cascaded codebook, we derive an upper bound on ergodic rate loss with maximum ratio transmission and show that the rate loss can be cut down by optimizing the feedback bit allocation during codebook generation. To minimize the upper bound, we propose a bit partitioning strategy that is adaptive to diverse environment and system parameters. Extensive simulations are presented to show the superiority and robustness of the cascaded codebook and the efficiency of the adaptive bit partitioning scheme.

DualLip: A System for Joint Lip Reading and Generation

Lip reading aims to recognize text from talking lip, while lip generation aims to synthesize talking lip according to text, which is a key component in talking face generation and is a dual task of lip reading. In this paper, we develop DualLip, a system that jointly improves lip reading and generation by leveraging the task duality and using unlabeled text and lip video data. The key ideas of the DualLip include: 1) Generate lip video from unlabeled text with a lip generation model, and use the pseudo pairs to improve lip reading; 2) Generate text from unlabeled lip video with a lip reading model, and use the pseudo pairs to improve lip generation. We further extend DualLip to talking face generation with two additionally introduced components: lip to face generation and text to speech generation. Experiments on GRID and TCD-TIMIT demonstrate the effectiveness of DualLip on improving lip reading, lip generation, and talking face generation by utilizing unlabeled data. Specifically, the lip generation model in our DualLip system trained with only10% paired data surpasses the performance of that trained with the whole paired data. And on the GRID benchmark of lip reading, we achieve 1.16% character error rate and 2.71% word error rate, outperforming the state-of-the-art models using the same amount of paired data.

Angle-Dependent Phase Shifter Model for Reconfigurable Intelligent Surfaces: Does the Angle-Reciprocity Hold?

The existing phase shifter models adopted for reconfigurable intelligent surfaces (RISs) have ignored the electromagnetic (EM) waves propagation behavior, thus cannot reveal practical effects of RIS on wireless communication systems. Based on the equivalent circuit, this paper introduces an angle-dependent phase shifter model for varactor-based RISs. To the best of our knowledge, this is the first phase shifter model which reveals that the incident angle of EM waves has influence on the reflection coefficient of RIS. In addition, the angle-reciprocity on RIS is investigated and further proved to be tenable when the reflection phase difference of adjacent RIS unit cells is invariant for an impinging EM wave and its reverse incident one. The angle-dependent characteristic of RIS is verified through full-wave simulation. According to our analysis and the simulation results, we find that the angle-reciprocity of varactor-based RIS only holds under small incident angles of both forward and reverse incident EM waves, thus limits the channel reciprocity in RIS-assisted TDD systems.