Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userPeek Inside the Closed World: Evaluating Autoencoder-Based Detection of DDoS to Cloud
97
0
0.0
(
0
)
Added by
Hang Guo
Publication date
2019
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
Machine-learning-based anomaly detection (ML-based AD) has been successful at detecting DDoS events in the lab. However published evaluations of ML-based AD have used only limited data and provided minimal insight into why it works. To address limited evaluation against real-world data, we apply autoencoder, an existing ML-AD model, to 57 DDoS attack events captured at 5 cloud IPs from a major cloud provider. We show that our models detect nearly all malicious flows for 2 of the 4 cloud IPs under attack (at least 99.99%) and detect most malicious flows (94.75% and 91.37%) for the remaining 2 IPs. Our models also maintain near-zero false positives on benign flows to all 5 IPs. Our primary contribution is to improve our understanding for why ML-based AD works on some malicious flows but not others. We interpret our detection results with feature attribution and counterfactual explanation. We show that our models are better at detecting malicious flows with anomalies on allow-listed features (those with only a few benign values) than flows with anomalies on deny-listed features (those with mostly benign values) because our models are more likely to learn correct normality for allow-listed features. We then show that our models are better at detecting malicious flows with anomalies on unordered features (that have no ordering among their values) than flows with anomalies on ordered features because even with incomplete normality, our models could still detect anomalies on unordered feature with high recall. Lastly, we summarize the implications of what we learn on applying autoencoder-based AD in production: training with noisy real-world data is possible, autoencoder can reliably detect real-world anomalies on well-represented unordered features and combinations of autoencoder-based AD and heuristic-based filters can help both.
rate research
Read More
Wireless sensor networks (WSN) are fundamental to the Internet of Things (IoT) by bridging the gap between the physical and the cyber worlds. Anomaly detection is a critical task in this context as it is responsible for identifying various events of interests such as equipment faults and undiscovered phenomena. However, this task is challenging because of the elusive nature of anomalies and the volatility of the ambient environments. In a resource-scarce setting like WSN, this challenge is further elevated and weakens the suitability of many existing solutions. In this paper, for the first time, we introduce autoencoder neural networks into WSN to solve the anomaly detection problem. We design a two-part algorithm that resides on sensors and the IoT cloud respectively, such that (i) anomalies can be detected at sensors in a fully distributed manner without the need for communicating with any other sensors or the cloud, and (ii) the relatively more computation-intensive learning task can be handled by the cloud with a much lower (and configurable) frequency. In addition to the minimal communication overhead, the computational load on sensors is also very low (of polynomial complexity) and readily affordable by most COTS sensors. Using a real WSN indoor testbed and sensor data collected over 4 consecutive months, we demonstrate via experiments that our proposed autoencoder-based anomaly detection mechanism achieves high detection accuracy and low false alarm rate. It is also able to adapt to unforeseeable and new changes in a non-stationary environment, thanks to the unsupervised learning feature of our chosen autoencoder neural networks.
IP Anycast is used for services such as DNS and Content Delivery Networks to provide the capacity to handle Distributed Denial-of-Service (DDoS) attacks. During a DDoS attack service operators may wish to redistribute traffic between anycast sites to take advantage of sites with unused or greater capacity. Depending on site traffic and attack size, operators may instead choose to concentrate attackers in a few sites to preserve operation in others. Previously service operators have taken these actions during attacks, but how to do so has not been described publicly. This paper meets that need, describing methods to use BGP to shift traffic when under DDoS that can build a response playbook. Operators can use this playbook, with our new method to estimate attack size, to respond to attacks. We also explore constraints on responses seen in an anycast deployment.
To effectively tackle the security threats towards the Internet of things, we propose a SOM-based DDoS defense mechanism using software-defined networking (SDN) in this paper. The main idea of the mechanism is to deploy a SDN-based gateway to protect the device services in the Internet of things. The gateway provides DDoS defense mechanism based on SOM neural network. By means of SOM-based DDoS defense mechanism, the gateway can effectively identify the malicious sensing devices in the IoT, and automatically block those malicious devices after detecting them, so that it can effectively enforce the security and robustness of the system when it is under DDoS attacks. In order to validate the feasibility and effectiveness of the mechanism, we leverage POX controller and Mininet emulator to implement an experimental system, and further implement the aforementioned security enforcement mechanisms with Python. The final experimental results illustrate that the mechanism is truly effective under the different test scenarios.
Distributed Denial-of-Service (DDoS) attacks are a major problem in the Internet today. In one form of a DDoS attack, a large number of compromised hosts send unwanted traffic to the victim, thus exhausting the resources of the victim and preventing it from serving its legitimate clients. One of the main mechanisms that have been proposed to deal with DDoS is filtering, which allows routers to selectively block unwanted traffic. Given the magnitude of DDoS attacks and the high cost of filters in the routers today, the successful mitigation of a DDoS attack using filtering crucially depends on the efficient allocation of filtering resources. In this paper, we consider a single router, typically the gateway of the victim, with a limited number of available filters. We study how to optimally allocate filters to attack sources, or entire domains of attack sources, so as to maximize the amount of good traffic preserved, under a constraint on the number of filters. We formulate the problem as an optimization problem and solve it optimally using dynamic programming, study the properties of the optimal allocation, experiment with a simple heuristic and evaluate our solutions for a range of realistic attack-scenarios. First, we look at a single-tier where the collateral damage is high due to the filtering at the granularity of domains. Second, we look at the two-tier problem where we have an additional constraint on the number of filters and the filtering is performed on the granularity of attackers and domains.
The problem of detecting anomalies in time series from network measurements has been widely studied and is a topic of fundamental importance. Many anomaly detection methods are based on packet inspection collected at the network core routers, with consequent disadvantages in terms of computational cost and privacy. We propose an alternative method in which packet header inspection is not needed. The method is based on the extraction of a normal subspace obtained by the tensor decomposition technique considering the correlation between different metrics. We propose a new approach for online tensor decomposition where changes in the normal subspace can be tracked efficiently. Another advantage of our proposal is the interpretability of the obtained models. The flexibility of the method is illustrated by applying it to two distinct examples, both using actual data collected on residential routers.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
