Proactive and AoI-aware Failure Recovery for Stateful NFV-enabled Zero-Touch 6G Networks: Model-Free DRL Approach

In this paper, we propose a Zero-Touch, deep reinforcement learning (DRL)-based Proactive Failure Recovery framework called ZT-PFR for stateful network function virtualization (NFV)-enabled networks. To this end, we formulate a resource-efficient optimization problem minimizing the network cost function including resource cost and wrong decision penalty. As a solution, we propose state-of-the-art DRL-based methods such as soft-actor-critic (SAC) and proximal-policy-optimization (PPO). In addition, to train and test our DRL agents, we propose a novel impending-failure model. Moreover, to keep network status information at an acceptable freshness level for appropriate decision-making, we apply the concept of age of information to strike a balance between the event and scheduling based monitoring. Several key systems and DRL algorithm design insights for ZT-PFR are drawn from our analysis and simulation results. For example, we use a hybrid neural network, consisting long short-term memory layers in the DRL agents structure, to capture impending-failures time dependency.

Robust Energy-Efficient Resource Management, SIC Ordering, and Beamforming Design for MC MISO-NOMA Enabled 6G

This paper studies a novel approach for successive interference cancellation (SIC) ordering and beamforming in a multiple antennas non-orthogonal multiple access (NOMA) network with multi-carrier multi-user setup. To this end, we formulate a joint beamforming design, subcarrier allocation, user association, and SIC ordering algorithm to maximize the worst-case energy efficiency (EE). The formulated problem is a non-convex mixed integer non-linear programming (MINLP) which is generally difficult to solve. To handle it, we first adopt the linearizion technique as well as relaxing the integer variables, and then we employ the Dinkelbach algorithm to convert it into a more mathematically tractable form. The adopted non-convex optimization problem is transformed into an equivalent rank-constrained semidefinite programming (SDP) and is solved by SDP relaxation and exploiting sequential fractional programming. Furthermore, to strike a balance between complexity and performance, a low complex approach based on alternative optimization is adopted. Numerical results unveil that the proposed SIC ordering method outperforms the conventional existing works addressed in the literature.

A Unified Framework for Joint Energy and AoI Optimization via Deep Reinforcement Learning for NOMA MEC-based Networks

In this paper, we design a novel scheduling and resource allocation algorithm for a smart mobile edge computing (MEC) assisted radio access network. Different from previous energy efficiency (EE) based or the average age of information (AAoI)-based network designs, we propose a unified metric for simultaneously optimizing ESE and AAoI of the network. To further improve the system capacity, non-orthogonal multiple access (NOMA) is proposed as a candidate for multiple access schemes for future cellular networks. Our main aim is to maximize the long-term objective function under AoI, NOMA, and resource capacity constraints using stochastic optimization. To overcome the complexities and unknown dynamics of the network parameters (e.g., wireless channel and interference), we apply the concept of reinforcement learning and implement a deep Q-network (DQN). Simulation results illustrate the effectiveness of the proposed framework and analyze different parameters impact on network performance. Based on the results, our proposed reward function converges fast with negligible loss value. Also, they illustrate our work outperforms the existing state of the art baselines up to 64% in the objective function and 51% in AAoI, which are stated as examples.

Joint Radio Resource Allocation and Cooperative Caching in PD-NOMA-Based HetNets

In this paper, we propose a novel joint resource allocation and cooperative caching scheme for power-domain non-orthogonal multiple access (PD-NOMA)-based heterogeneous networks (HetNets). In our scheme, the requested content is fetched directly from the edge if it is cached in the storage of one of the base stations (BSs), and otherwise is fetched via the backhaul. Our scheme consists of two phases: 1. Caching phase where the contents are saved in the storage of the BSs, and 2. Delivery phase where the requested contents are delivered to users. We formulate a novel optimization problem over radio resources and content placement variables. We aim to minimize the network cost subject to quality-of-service (QoS), caching, subcarrier assignment, and power allocation constraints. By exploiting advanced optimization methods, such as alternative search method (ASM), Hungarian algorithm, successive convex approximation (SCA), we obtain an efficient sub-optimal solution of the optimization problem. Numerical results illustrate that our ergodic caching policy via the proposed resource management algorithm can achieve a considerable reduction on the total cost on average compared to the most popular caching and random caching policy. Moreover, our cooperative NOMA scheme outperforms orthogonal multiple access (OMA) in terms of the delivery cost in general with an acceptable complexity increase.