Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such problems. In this paper, we first utilize single agent low-complex compound action actor-critic RL to cover both discrete and continuous actions and jointly minimize VNF cost and AoI in terms of network resources under end-to end Quality of Service constraints. To surmount the single-agent capacity limitation for learning, we then extend our solution to a multi-agent DRL scheme in which agents collaborate with each other. Simulation results demonstrate that single-agent schemes significantly outperform the greedy algorithm in terms of average network cost and AoI. Moreover, multi-agent solution decreases the average cost by dividing the tasks between the agents. However, it needs more iterations to be learned due to the requirement on the agents collaboration.

Delay-Distortion-Power Trade Offs in Quasi-Stationary Source Transmission over Block Fading Channels

This paper investigates delay-distortion-power trade offs in transmission of quasi-stationary sources over block fading channels by studying encoder and decoder buffering techniques to smooth out the source and channel variations. Four source and channel coding schemes that consider buffer and power constraints are presented to minimize the reconstructed source distortion. The first one is a high performance scheme, which benefits from optimized source and channel rate adaptation. In the second scheme, the channel coding rate is fixed and optimized along with transmission power with respect to channel and source variations; hence this scheme enjoys simplicity of implementation. The two last schemes have fixed transmission power with optimized adaptive or fixed channel coding rate. For all the proposed schemes, closed form solutions for mean distortion, optimized rate and power are provided and in the high SNR regime, the mean distortion exponent and the asymptotic mean power gains are derived. The proposed schemes with buffering exploit the diversity due to source and channel variations. Specifically, when the buffer size is limited, fixed channel rate adaptive power scheme outperforms an adaptive rate fixed power scheme. Furthermore, analytical and numerical results demonstrate that with limited buffer size, the system performance in terms of reconstructed signal SNR saturates as transmission power is increased, suggesting that appropriate buffer size selection is important to achieve a desired reconstruction quality.