Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation

The conventional federated learning (FedL) architecture distributes machine learning (ML) across worker devices by having them train local models that are periodically aggregated by a server. FedL ignores two important characteristics of contemporary wireless networks, however: (i) the network may contain heterogeneous communication/computation resources, while (ii) there may be significant overlaps in devices local data distributions. In this work, we develop a novel optimization methodology that jointly accounts for these factors via intelligent device sampling complemented by device-to-device (D2D) offloading. Our optimization aims to select the best combination of sampled nodes and data offloading configuration to maximize FedL training accuracy subject to realistic constraints on the network topology and device capabilities. Theoretical analysis of the D2D offloading subproblem leads to new FedL convergence bounds and an efficient sequential convex optimizer. Using this result, we develop a sampling methodology based on graph convolutional networks (GCNs) which learns the relationship between network attributes, sampled nodes, and resulting offloading that maximizes FedL accuracy. Through evaluation on real-world datasets and network measurements from our IoT testbed, we find that our methodology while sampling less than 5% of all devices outperforms conventional FedL substantially both in terms of trained model accuracy and required resource utilization.

A Fast Graph Neural Network-Based Method for Winner Determination in Multi-Unit Combinatorial Auctions

The combinatorial auction (CA) is an efficient mechanism for resource allocation in different fields, including cloud computing. It can obtain high economic efficiency and user flexibility by allowing bidders to submit bids for combinations of different items instead of only for individual items. However, the problem of allocating items among the bidders to maximize the auctioneers revenue, i.e., the winner determination problem (WDP), is NP-complete to solve and inapproximable. Existing works for WDPs are generally based on mathematical optimization techniques and most of them focus on the single-unit WDP, where each item only has one unit. On the contrary, few works consider the multi-unit WDP in which each item may have multiple units. Given that the multi-unit WDP is more complicated but prevalent in cloud computing, we propose leveraging machine learning (ML) techniques to develop a novel low-complexity algorithm for solving this problem with negligible revenue loss. Specifically, we model the multi-unit WDP as an augmented bipartite bid-item graph and use a graph neural network (GNN) with half-convolution operations to learn the probability of each bid belonging to the optimal allocation. To improve the sample generation efficiency and decrease the number of needed labeled instances, we propose two different sample generation processes. We also develop two novel graph-based post-processing algorithms to transform the outputs of the GNN into feasible solutions. Through simulations on both synthetic instances and a specific virtual machine (VM) allocation problem in a cloud computing platform, we validate that our proposed method can approach optimal performance with low complexity and has good generalization ability in terms of problem size and user-type distribution.

Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks

Federated learning has generated significant interest, with nearly all works focused on a ``star topology where nodes/devices are each connected to a central server. We migrate away from this architecture and extend it through the network dimension to the case where there are multiple layers of nodes between the end devices and the server. Specifically, we develop multi-stage hybrid federated learning (MH-FL), a hybrid of intra- and inter-layer model learning that considers the network as a multi-layer cluster-based structure, each layer of which consists of multiple device clusters. MH-FL considers the topology structures among the nodes in the clusters, including local networks formed via device-to-device (D2D) communications. It orchestrates the devices at different network layers in a collaborative/cooperative manner (i.e., using D2D interactions) to form local consensus on the model parameters, and combines it with multi-stage parameter relaying between layers of the tree-shaped hierarchy. We derive the upper bound of convergence for MH-FL with respect to parameters of the network topology (e.g., the spectral radius) and the learning algorithm (e.g., the number of D2D rounds in different clusters). We obtain a set of policies for the D2D rounds at different clusters to guarantee either a finite optimality gap or convergence to the global optimum. We then develop a distributed control algorithm for MH-FL to tune the D2D rounds in each cluster over time to meet specific convergence criteria. Our experiments on real-world datasets verify our analytical results and demonstrate the advantages of MH-FL in terms of resource utilization metrics.

From Federated to Fog Learning: Distributed Machine Learning over Heterogeneous Wireless Networks

Machine learning (ML) tasks are becoming ubiquitous in todays network applications. Federated learning has emerged recently as a technique for training ML models at the network edge by leveraging processing capabilities across the nodes that collect the data. There are several challenges with employing conventional federated learning in contemporary networks, due to the significant heterogeneity in compute and communication capabilities that exist across devices. To address this, we advocate a new learning paradigm called fog learning which will intelligently distribute ML model training across the continuum of nodes from edge devices to cloud servers. Fog learning enhances federated learning along three major dimensions: network, heterogeneity, and proximity. It considers a multi-layer hybrid learning framework consisting of heterogeneous devices with various proximities. It accounts for the topology structures of the local networks among the heterogeneous nodes at each network layer, orchestrating them for collaborative/cooperative learning through device-to-device (D2D) communications. This migrates from star network topologies used for parameter transfers in federated learning to more distributed topologies at scale. We discuss several open research directions to realizing fog learning.

Lifetime Maximization for UAV-assisted Data Gathering Networks in the Presence of Jamming

Deployment of unmanned aerial vehicles (UAVs) is recently getting significant attention due to a variety of practical use cases, such as surveillance, data gathering, and commodity delivery. Since UAVs are powered by batteries, energy efficient communication is of paramount importance. In this paper, we investigate the problem of lifetime maximization of a UAV-assisted network in the presence of multiple sources of interference, where the UAVs are deployed to collect data from a set of wireless sensors. We demonstrate that the placement of the UAVs play a key role in prolonging the lifetime of the network since the required transmission powers of the UAVs are closely related to their locations in space. In the proposed scenario, the UAVs transmit the gathered data to a primary UAV called textit{leader}, which is in charge of forwarding the data to the base station (BS) via a backhaul UAV network. We deploy tools from spectral graph theory to tackle the problem due to its high non-convexity. Simulation results demonstrate that our proposed method can significantly improve the lifetime of the UAV network.

Energy-Aware Stochastic UAV-Assisted Surveillance

With the ease of deployment, capabilities of evading the jammers and obscuring their existence, unmanned aerial vehicles (UAVs) are one of the most suitable candidates to perform surveillance. There exists a body of literature in which the inspectors follow a deterministic trajectory to conduct surveillance, which results in a predictable environment for malicious entities. Thus, introducing randomness to the surveillance is of particular interest. In this work, we propose a novel framework for stochastic UAV-assisted surveillance that i) inherently considers the battery constraints of the UAVs, ii) proposes random moving patterns modeled via random walks, and iii) adds another degree of randomness to the system via considering probabilistic inspections. We formulate the problem of interest, i.e., obtaining the energy-efficient random walk and inspection policies of the UAVs subject to probabilistic constraints on inspection criteria of the sites and battery consumption of the UAVs, which turns out to be signomial programming that is highly non-convex. To solve it, we propose a centralized and a distributed algorithm along with their performance guarantee. This work contributes to both UAV-assisted surveillance and classic random walk literature by designing random walks with random inspection policies on weighted graphs with energy limited random walkers.

Energy-aware Allocation of Graph Jobs in Vehicular Cloud Computing-enabled Software-defined IoV

Software-defined internet of vehicles (SDIoV) has emerged as a promising paradigm to realize flexible and comprehensive resource management, for next generation automobile transportation systems. In this paper, a vehicular cloud computing-based SDIoV framework is studied wherein the joint allocation of transmission power and graph job is formulated as a nonlinear integer programming problem. To effectively address the problem, a structure-preservation-based two-stage allocation scheme is proposed that decouples template searching from power allocation. Specifically, a hierarchical tree-based random subgraph isomorphism mechanism is applied in the first stage by identifying potential mappings (templates) between the components of graph jobs and service providers. A structure-preserving simulated annealing-based power allocation algorithm is adopted in the second stage to achieve the trade-off between the job completion time and energy consumption. Extensive simulations are conducted to verify the performance of the proposed algorithms.

A Truthful Auction for Graph Job Allocation in Vehicular Cloud-assisted Networks

Vehicular cloud computing has emerged as a promising solution to fulfill users demands on processing computation-intensive applications in modern driving environments. Such applications are commonly represented by graphs consisting of components and edges. However, encouraging vehicles to share resources poses significant challenges owing to users selfishness. In this paper, an auction-based graph job allocation problem is studied in vehicular cloud-assisted networks considering resource reutilization. Our goal is to map each buyer (component) to a feasible seller (virtual machine) while maximizing the buyers utility-of-service, which concerns the execution time and commission cost. First, we formulate the auction-based graph job allocation as an integer programming (IP) problem. Then, a Vickrey-Clarke-Groves based payment rule is proposed which satisfies the desired economical properties, truthfulness and individual rationality. We face two challenges: 1) the above-mentioned IP problem is NP-hard; 2) one constraint associated with the IP problem poses addressing the subgraph isomorphism problem. Thus, obtaining the optimal solution is practically infeasible in large-scale networks. Motivated by which, we develop a structure-preserved matching algorithm by maximizing the utility-of-service-gain, and the corresponding payment rule which offers economical properties and low computation complexity. Extensive simulations demonstrate that the proposed algorithm outperforms the benchmark methods considering various problem sizes.

Optimal Jammer Placement in UAV-assisted Relay Networks

We consider the relaying application of unmanned aerial vehicles (UAVs), in which UAVs are placed between two transceivers (TRs) to increase the throughput of the system. Instead of studying the placement of UAVs as pursued in existing literature, we focus on investigating the placement of a jammer or a major source of interference on the ground to effectively degrade the performance of the system, which is measured by the maximum achievable data rate of transmission between the TRs. We demonstrate that the optimal placement of the jammer is in general a non-convex optimization problem, for which obtaining the solution directly is intractable. Afterward, using the inherent characteristics of the signal-to-interference ratio (SIR) expressions, we propose a tractable approach to find the optimal position of the jammer. Based on the proposed approach, we investigate the optimal positioning of the jammer in both dual-hop and multi-hop UAV relaying settings. Numerical simulations are provided to evaluate the performance of our proposed method.

Multi-Task Offloading over Vehicular Clouds under Graph-based Representation

Vehicular cloud computing has emerged as a promising paradigm for realizing user requirements in computation-intensive tasks in modern driving environments. In this paper, a novel framework of multi-task offloading over vehicular clouds (VCs) is introduced where tasks and VCs are modeled as undirected weighted graphs. Aiming to achieve a trade-off between minimizing task completion time and data exchange costs, task components are efficiently mapped to available virtual machines in the related VCs. The problem is formulated as a non-linear integer programming problem, mainly under constraints of limited contact between vehicles as well as available resources, and addressed in low-traffic and rush-hour scenarios. In low-traffic cases, we determine optimal solutions; in rush-hour cases, a connection-restricted randommatching-based subgraph isomorphism algorithm is proposed that presents low computational complexity. Evaluations of the proposed algorithms against greedy-based baseline methods are conducted via extensive simulations.