اشترك بالحزمة الذهبية واحصل على وصول غير محدود شمرا أكاديميا
تسجيل مستخدم جديدPath Design for Cellular-Connected UAV with Reinforcement Learning
142
0
0.0
(
0
)
اسأل ChatGPT حول البحث
ﻻ يوجد ملخص باللغة العربية
This paper studies the path design problem for cellular-connected unmanned aerial vehicle (UAV), which aims to minimize its mission completion time while maintaining good connectivity with the cellular network. We first argue that the conventional path design approach via formulating and solving optimization problems faces several practical challenges, and then propose a new reinforcement learning-based UAV path design algorithm by applying emph{temporal-difference} method to directly learn the emph{state-value function} of the corresponding Markov Decision Process. The proposed algorithm is further extended by using linear function approximation with tile coding to deal with large state space. The proposed algorithms only require the raw measured or simulation-generated signal strength as the input and are suitable for both online and offline implementations. Numerical results show that the proposed path designs can successfully avoid the coverage holes of cellular networks even in the complex urban environment.
قيم البحث
اقرأ أيضاً
In this paper, we study the three-dimensional (3D) path planning for a cellular-connected unmanned aerial vehicle (UAV) to minimize its flying distance from given initial to final locations, while ensuring a target link quality in terms of the expect
ed signal-to-interference-plus-noise ratio (SINR) at the UAV receiver with each of its associated ground base stations (GBSs) during the flight. To exploit the location-dependent and spatially varying channel as well as interference over the 3D space, we propose a new radio map based path planning framework for the UAV. Specifically, we consider the channel gain map of each GBS that provides its large-scale channel gains with uniformly sampled locations on a 3D grid, which are due to static and large-size obstacles (e.g., buildings) and thus assumed to be time-invariant. Based on the channel gain maps of GBSs as well as their loading factors, we then construct an SINR map that depicts the expected SINR levels over the sampled 3D locations. By leveraging the obtained SINR map, we proceed to derive the optimal UAV path by solving an equivalent shortest path problem (SPP) in graph theory. We further propose a grid quantization approach where the grid points in the SINR map are more coarsely sampled by exploiting the spatial channel/interference correlation over neighboring grids. Then, we solve an approximate SPP over the reduced-size SINR map (graph) with reduced complexity. Numerical results show that the proposed solution can effectively minimize the flying distance/time of the UAV subject to its communication quality constraint, and a flexible trade-off between performance and complexity can be achieved by adjusting the grid quantization ratio in the SINR map. Moreover, the proposed solution significantly outperforms various benchmark schemes without fully exploiting the channel/interference spatial distribution in the network.
A mega-constellation of low-altitude earth orbit (LEO) satellites (SATs) and burgeoning unmanned aerial vehicles (UAVs) are promising enablers for high-speed and long-distance communications in beyond fifth-generation (5G) systems. Integrating SATs a
nd UAVs within a non-terrestrial network (NTN), in this article we investigate the problem of forwarding packets between two faraway ground terminals through SAT and UAV relays using either millimeter-wave (mmWave) radio-frequency (RF) or free-space optical (FSO) link. Towards maximizing the communication efficiency, the real-time associations with orbiting SATs and the moving trajectories of UAVs should be optimized with suitable FSO/RF links, which is challenging due to the time-varying network topology and a huge number of possible control actions. To overcome the difficulty, we lift this problem to multi-agent deep reinforcement learning (MARL) with a novel action dimensionality reduction technique. Simulation results corroborate that our proposed SAT-UAV integrated scheme achieves 1.99x higher end-to-end sum throughput compared to a benchmark scheme with fixed ground relays. While improving the throughput, our proposed scheme also aims to reduce the UAV control energy, yielding 2.25x higher energy efficiency than a baseline method only maximizing the throughput. Lastly, thanks to utilizing hybrid FSO/RF links, the proposed scheme achieves up to 62.56x higher peak throughput and 21.09x higher worst-case throughput than the cases utilizing either RF or FSO links, highlighting the importance of co-designing SAT-UAV associations, UAV trajectories, and hybrid FSO/RF links in beyond-5G NTNs.
In this paper, we study the path planning for a cellular-connected unmanned aerial vehicle (UAV) to minimize its flying distance from given initial to final locations, while ensuring a target link quality in terms of the large-scale channel gain with
each of its associated ground base stations (GBSs) during the flight. To this end, we propose the use of radio map that provides the information on the large-scale channel gains between each GBS and uniformly sampled locations on a three-dimensional (3D) grid over the region of interest, which are assumed to be time-invariant due to the generally static and large-size obstacles therein (e.g., buildings). Based on the given radio maps of the GBSs, we first obtain the optimal UAV path by solving an equivalent shortest path problem (SPP) in graph theory. To reduce the computation complexity of the optimal solution, we further propose a grid quantization method whereby the grid points in each GBSs radio map are more coarsely sampled by exploiting the spatial channel correlation over neighboring grids. Then, we solve the approximate SPP over the reduced-size radio map (graph) more efficiently. Numerical results show that the proposed solutions can effectively minimize the flying distance of the UAV subject to its communication quality constraint. Moreover, a flexible trade-off between performance and complexity can be achieved by adjusting the quantization ratio for the radio map.
In this paper, we study the trajectory design for a cellular-connected unmanned aerial vehicle (UAV) with given initial and final locations, while communicating with the ground base stations (GBSs) along its flight. We consider delay-limited communic
ations between the UAV and its associated GBSs, where a given signal-to-noise ratio (SNR) target needs to be satisfied at the receiver. However, in practice, due to various factors such as quality-of-service (QoS) requirement, GBSs availability and UAV mobility constraints, the SNR target may not be met at certain time periods during the flight, each termed as an outage duration. In this paper, we aim to optimize the UAV trajectory to minimize its mission completion time, subject to a constraint on the maximum tolerable outage duration in its flight. To tackle this non-convex problem, we first transform it into a more tractable form and thereby reveal some useful properties of the optimal trajectory solution. Based on these properties, we then further simplify the problem and propose efficient algorithms to check the feasibility of the problem as well as to obtain its optimal and high-quality suboptimal solutions, by leveraging graph theory and convex optimization techniques. Numerical results show that our proposed trajectory designs outperform the conventional method based on dynamic programming, in terms of both performance and complexity.
In cellular-connected unmanned aerial vehicle (UAV) network, a minimization problem on the weighted sum of time cost and expected outage duration is considered. Taking advantage of UAVs adjustable mobility, an intelligent UAV navigation approach is f
ormulated to achieve the aforementioned optimization goal. Specifically, after mapping the navigation task into a Markov decision process (MDP), a deep reinforcement learning (DRL) solution with novel quantum-inspired experience replay (QiER) framework is proposed to help the UAV find the optimal flying direction within each time slot, and thus the designed trajectory towards the destination can be generated. Via relating experienced transitions importance to its associated quantum bit (qubit) and applying Grover iteration based amplitude amplification technique, the proposed DRL-QiER solution can commit a better trade-off between sampling priority and diversity. Compared to several representative baselines, the effectiveness and supremacy of the proposed DRL-QiER solution are demonstrated and validated in numerical results.
الأسئلة المقترحة
سجل دخول لتتمكن من نشر تعليقات
التعليقات
جاري جلب التعليقات
سجل دخول لتتمكن من متابعة معايير البحث التي قمت باختيارها
