Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userPath Planning for Cellular-Connected UAV: A DRL Solution with Quantum-Inspired Experience Replay
129
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
In cellular-connected unmanned aerial vehicle (UAV) network, a minimization problem on the weighted sum of time cost and expected outage duration is considered. Taking advantage of UAVs adjustable mobility, an intelligent UAV navigation approach is formulated to achieve the aforementioned optimization goal. Specifically, after mapping the navigation task into a Markov decision process (MDP), a deep reinforcement learning (DRL) solution with novel quantum-inspired experience replay (QiER) framework is proposed to help the UAV find the optimal flying direction within each time slot, and thus the designed trajectory towards the destination can be generated. Via relating experienced transitions importance to its associated quantum bit (qubit) and applying Grover iteration based amplitude amplification technique, the proposed DRL-QiER solution can commit a better trade-off between sampling priority and diversity. Compared to several representative baselines, the effectiveness and supremacy of the proposed DRL-QiER solution are demonstrated and validated in numerical results.
rate research
Read More
In this paper, we study the three-dimensional (3D) path planning for a cellular-connected unmanned aerial vehicle (UAV) to minimize its flying distance from given initial to final locations, while ensuring a target link quality in terms of the expected signal-to-interference-plus-noise ratio (SINR) at the UAV receiver with each of its associated ground base stations (GBSs) during the flight. To exploit the location-dependent and spatially varying channel as well as interference over the 3D space, we propose a new radio map based path planning framework for the UAV. Specifically, we consider the channel gain map of each GBS that provides its large-scale channel gains with uniformly sampled locations on a 3D grid, which are due to static and large-size obstacles (e.g., buildings) and thus assumed to be time-invariant. Based on the channel gain maps of GBSs as well as their loading factors, we then construct an SINR map that depicts the expected SINR levels over the sampled 3D locations. By leveraging the obtained SINR map, we proceed to derive the optimal UAV path by solving an equivalent shortest path problem (SPP) in graph theory. We further propose a grid quantization approach where the grid points in the SINR map are more coarsely sampled by exploiting the spatial channel/interference correlation over neighboring grids. Then, we solve an approximate SPP over the reduced-size SINR map (graph) with reduced complexity. Numerical results show that the proposed solution can effectively minimize the flying distance/time of the UAV subject to its communication quality constraint, and a flexible trade-off between performance and complexity can be achieved by adjusting the grid quantization ratio in the SINR map. Moreover, the proposed solution significantly outperforms various benchmark schemes without fully exploiting the channel/interference spatial distribution in the network.
In this paper, we consider a wireless uplink transmission scenario in which an unmanned aerial vehicle (UAV) serves as an aerial base station collecting data from ground users. To optimize the expected sum uplink transmit rate without any prior knowledge of ground users (e.g., locations, channel state information and transmit power), the trajectory planning problem is optimized via the quantum-inspired reinforcement learning (QiRL) approach. Specifically, the QiRL method adopts novel probabilistic action selection policy and new reinforcement strategy, which are inspired by the collapse phenomenon and amplitude amplification in quantum computation theory, respectively. Numerical results demonstrate that the proposed QiRL solution can offer natural balancing between exploration and exploitation via ranking collapse probabilities of possible actions, compared to the traditional reinforcement learning approaches which are highly dependent on tuned exploration parameters.
In this paper, we study the path planning for a cellular-connected unmanned aerial vehicle (UAV) to minimize its flying distance from given initial to final locations, while ensuring a target link quality in terms of the large-scale channel gain with each of its associated ground base stations (GBSs) during the flight. To this end, we propose the use of radio map that provides the information on the large-scale channel gains between each GBS and uniformly sampled locations on a three-dimensional (3D) grid over the region of interest, which are assumed to be time-invariant due to the generally static and large-size obstacles therein (e.g., buildings). Based on the given radio maps of the GBSs, we first obtain the optimal UAV path by solving an equivalent shortest path problem (SPP) in graph theory. To reduce the computation complexity of the optimal solution, we further propose a grid quantization method whereby the grid points in each GBSs radio map are more coarsely sampled by exploiting the spatial channel correlation over neighboring grids. Then, we solve the approximate SPP over the reduced-size radio map (graph) more efficiently. Numerical results show that the proposed solutions can effectively minimize the flying distance of the UAV subject to its communication quality constraint. Moreover, a flexible trade-off between performance and complexity can be achieved by adjusting the quantization ratio for the radio map.
This paper studies the path design problem for cellular-connected unmanned aerial vehicle (UAV), which aims to minimize its mission completion time while maintaining good connectivity with the cellular network. We first argue that the conventional path design approach via formulating and solving optimization problems faces several practical challenges, and then propose a new reinforcement learning-based UAV path design algorithm by applying emph{temporal-difference} method to directly learn the emph{state-value function} of the corresponding Markov Decision Process. The proposed algorithm is further extended by using linear function approximation with tile coding to deal with large state space. The proposed algorithms only require the raw measured or simulation-generated signal strength as the input and are suitable for both online and offline implementations. Numerical results show that the proposed path designs can successfully avoid the coverage holes of cellular networks even in the complex urban environment.
In this paper, a novel training paradigm inspired by quantum computation is proposed for deep reinforcement learning (DRL) with experience replay. In contrast to traditional experience replay mechanism in DRL, the proposed deep reinforcement learning with quantum-inspired experience replay (DRL-QER) adaptively chooses experiences from the replay buffer according to the complexity and the replayed times of each experience (also called transition), to achieve a balance between exploration and exploitation. In DRL-QER, transitions are first formulated in quantum representations, and then the preparation operation and the depreciation operation are performed on the transitions. In this progress, the preparation operation reflects the relationship between the temporal difference errors (TD-errors) and the importance of the experiences, while the depreciation operation is taken into account to ensure the diversity of the transitions. The experimental results on Atari 2600 games show that DRL-QER outperforms state-of-the-art algorithms such as DRL-PER and DCRL on most of these games with improved training efficiency, and is also applicable to such memory-based DRL approaches as double network and dueling network.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
