اشترك بالحزمة الذهبية واحصل على وصول غير محدود شمرا أكاديميا
تسجيل مستخدم جديدReinforcement Learning Random Access for Delay-Constrained Heterogeneous Wireless Networks: A Two-User Case
217
0
0.0
(
0
)
اسأل ChatGPT حول البحث
ﻻ يوجد ملخص باللغة العربية
In this paper, we investigate the random access problem for a delay-constrained heterogeneous wireless network. As a first attempt to study this new problem, we consider a network with two users who deliver delay-constrained traffic to an access point (AP) via a common unreliable collision wireless channel. We assume that one user (called user 1) adopts ALOHA and we optimize the random access scheme of the other user (called user 2). The most intriguing part of this problem is that user 2 does not know the information of user 1 but needs to maximize the system timely throughput. Such a paradigm of collaboratively sharing spectrum is envisioned by DARPA to better dynamically match the supply and demand in the future [1], [2]. We first propose a Markov Decision Process (MDP) formulation to derive a modelbased upper bound, which can quantify the performance gap of any designed schemes. We then utilize reinforcement learning (RL) to design an R-learning-based [3]-[5] random access scheme, called TSRA. We finally carry out extensive simulations to show that TSRA achieves close-to-upper-bound performance and better performance than the existing baseline DLMA [6], which is our counterpart scheme for delay-unconstrained heterogeneous wireless network. All source code is publicly available in https://github.com/DanzhouWu/TSRA.
قيم البحث
اقرأ أيضاً
As low frequency band becomes more and more crowded, millimeter-wave (mmWave) has attracted significant attention recently. IEEE has released the 802.11ad standard to satisfy the demand of ultra-high-speed communication. It adopts beamforming technol
ogy that can generate directional beams to compensate for high path loss. In the Association Beamforming Training (A-BFT) phase of beamforming (BF) training, a station (STA) randomly selects an A-BFT slot to contend for training opportunity. Due to the limited number of A-BFT slots, A-BFT phase suffers high probability of collisions in dense user scenarios, resulting in inefficient training performance. Based on the evaluation of the IEEE 802.11ad standard and 802.11ay draft in dense user scenarios of mmWave wireless networks, we propose an enhanced A-BFT beam training and random access mechanism, including the Separated A-BFT (SA-BFT) and Secondary Backoff A-BFT (SBA-BFT). The SA-BFT can provide more A-BFT slots and divide A-BFT slots into two regions by defining a new `E-A-BFT Length field compared to the legacy 802.11ad A-BFT, thereby maintaining compatibility when 802.11ay devices are mixed with 802.11ad devices. It can also reduce the collision probability in dense user scenarios greatly. The SBA-BFT performs secondary backoff with very small overhead of transmission opportunities within one A-BFT slot, which not only further reduces collision probability, but also improves the A-BFT slots utilization. Furthermore, we propose a three-dimensional Markov model to analyze the performance of the SBA-BFT. The analytical and simulation results show that both the SA-BFT and the SBA-BFT can significantly improve BF training efficiency, which are beneficial to the optimization design of dense user wireless networks based on the IEEE 802.11ay standard and mmWave technology.
This letter proposes a novel random medium access control (MAC) based on a transmission opportunity prediction, which can be measured in a form of a conditional success probability given transmitter-side interference. A transmission probability depen
ds on the opportunity prediction, preventing indiscriminate transmissions and reducing excessive interference causing collisions. Using stochastic geometry, we derive a fixed-point equation to provide the optimal transmission probability maximizing a proportionally fair throughput. Its approximated solution is given in closed form. The proposed MAC is applicable to full-duplex networks, leading to significant throughput improvement by allowing more nodes to transmit.
In this paper, we propose a two-layer framework to learn the optimal handover (HO) controllers in possibly large-scale wireless systems supporting mobile Internet-of-Things (IoT) users or traditional cellular users, where the user mobility patterns c
ould be heterogeneous. In particular, our proposed framework first partitions the user equipments (UEs) with different mobility patterns into clusters, where the mobility patterns are similar in the same cluster. Then, within each cluster, an asynchronous multi-user deep reinforcement learning scheme is developed to control the HO processes across the UEs in each cluster, in the goal of lowering the HO rate while ensuring certain system throughput. In this scheme, we use a deep neural network (DNN) as an HO controller learned by each UE via reinforcement learning in a collaborative fashion. Moreover, we use supervised learning in initializing the DNN controller before the execution of reinforcement learning to exploit what we already know with traditional HO schemes and to mitigate the negative effects of random exploration at the initial stage. Furthermore, we show that the adopted global-parameter-based asynchronous framework enables us to train faster with more UEs, which could nicely address the scalability issue to support large systems. Finally, simulation results demonstrate that the proposed framework can achieve better performance than the state-of-art on-line schemes, in terms of HO rates.
LoRa wireless networks are considered as a key enabling technology for next generation internet of things (IoT) systems. New IoT deployments (e.g., smart city scenarios) can have thousands of devices per square kilometer leading to huge amount of pow
er consumption to provide connectivity. In this paper, we investigate green LoRa wireless networks powered by a hybrid of the grid and renewable energy sources, which can benefit from harvested energy while dealing with the intermittent supply. This paper proposes resource management schemes of the limited number of channels and spreading factors (SFs) with the objective of improving the LoRa gateway energy efficiency. First, the problem of grid power consumption minimization while satisfying the systems quality of service demands is formulated. Specifically, both scenarios the uncorrelated and time-correlated channels are investigated. The optimal resource management problem is solved by decoupling the formulated problem into two sub-problems: channel and SF assignment problem and energy management problem. Since the optimal solution is obtained with high complexity, online resource management heuristic algorithms that minimize the grid energy consumption are proposed. Finally, taking into account the channel and energy correlation, adaptable resource management schemes based on Reinforcement Learning (RL), are developed. Simulations results show that the proposed resource management schemes offer efficient use of renewable energy in LoRa wireless networks.
Traditional concept of cognitive radio is the coexistence of primary and secondary user in multiplexed manner. we consider the opportunistic channel access scheme in IEEE 802.11 based networks subject to the interference mitigation scenario. Accordin
g to the protocol rule and due to the constraint of message passing, secondary user is unaware of the exact state of the primary user. In this paper, we have proposed an online algorithm for the secondary which assist determining a backoff counter or the decision of being idle for utilizing the time/frequency slot unoccupied by the primary user. Proposed algorithm is based on conventional reinforcement learning technique namely Q-Learning. Simulation has been conducted in order to prove the strength of this algorithm and also results have been compared with our contemporary solution of this problem where secondary user is aware of some states of primary user.
سجل دخول لتتمكن من نشر تعليقات
التعليقات
جاري جلب التعليقات
سجل دخول لتتمكن من متابعة معايير البحث التي قمت باختيارها
