Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCatch Me if I Can: Detecting Strategic Behaviour in Peer Assessment
83
0
0.0
(
0
)
Added by
Ivan Stelmakh
Publication date
2020
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
We consider the issue of strategic behaviour in various peer-assessment tasks, including peer grading of exams or homeworks and peer review in hiring or promotions. When a peer-assessment task is competitive (e.g., when students are graded on a curve), agents may be incentivized to misreport evaluations in order to improve their own final standing. Our focus is on designing methods for detection of such manipulations. Specifically, we consider a setting in which agents evaluate a subset of their peers and output rankings that are later aggregated to form a final ordering. In this paper, we investigate a statistical framework for this problem and design a principled test for detecting strategic behaviour. We prove that our test has strong false alarm guarantees and evaluate its detection ability in practical settings. For this, we design and execute an experiment that elicits strategic behaviour from subjects and release a dataset of patterns of strategic behaviour that may be of independent interest. We then use the collected data to conduct a series of real and semi-synthetic evaluations that demonstrate a strong detection power of our test.
rate research
Read More
With more than 500 million daily tweets from over 330 million active users, Twitter constantly attracts malicious users aiming to carry out phishing and malware-related attacks against its user base. It therefore becomes of paramount importance to assess the effectiveness of Twitters use of blacklists in protecting its users from such threats. We collected more than 182 million public tweets containing URLs from Twitters Stream API over a 2-month period and compared these URLs against 3 popular phishing, social engineering, and malware blacklists, including Google Safe Browsing (GSB). We focus on the delay period between an attack URL first being tweeted to appearing on a blacklist, as this is the timeframe in which blacklists do not warn users, leaving them vulnerable. Experiments show that, whilst GSB is effective at blocking a number of social engineering and malicious URLs within 6 hours of being tweeted, a significant number of URLs go undetected for at least 20 days. For instance, during one month, we discovered 4,930 tweets containing URLs leading to social engineering websites that had been tweeted to over 131 million Twitter users. We also discovered 1,126 tweets containing 376 blacklisted Bitly URLs that had a combined total of 991,012 clicks, posing serious security and privacy threats. In addition, an equally large number of URLs contained within public tweets remain in GSB for at least 150 days, raising questions about potential false positives in the blacklist. We also provide evidence to suggest that Twitter may no longer be using GSB to protect its users.
Direct scattering transform of nonlinear wave fields with solitons may lead to anomalous numerical errors of soliton phase and position parameters. With the focusing one-dimensional nonlinear Schrodinger equation serving as a model, we investigate this fundamental issue theoretically. Using the dressing method we find the landscape of soliton scattering coefficients in the plane of the complex spectral parameter for multi-soliton wave fields truncated within a finite domain, allowing us to capture the nature of particular numerical errors. They depend on the size of the computational domain $L$ leading to a counterintuitive exponential divergence when increasing $L$ in the presence of a small uncertainty in soliton eigenvalues. In contrast to classical textbooks, we reveal how one of the scattering coefficients loses its analytical properties due to the lack of the wave field compact support in case of $L to infty$. Finally, we demonstrate that despite this inherit direct scattering transform feature, the wave fields of arbitrary complexity can be reliably analysed.
We report that Ly$alpha$-emitting galaxies (LAEs) may not faithfully trace the cosmic web of neutral hydrogen (HI), but their distribution is likely biased depending on the viewing direction. We calculate the cross-correlation (CCF) between galaxies and Ly$alpha$ forest transmission fluctuations on the near and far sides of the galaxies separately, for three galaxy samples at $zsim2$: LAEs, [OIII] emitters (O3Es), and continuum-selected galaxies. We find that only LAEs have anisotropic CCFs, with the near side one showing lower signals up to $r=3-4~h^{-1}$ comoving Mpc. This means that the average HI density on the near side of LAEs is lower than that on the far-side by a factor of $2.1$ under the Fluctuating Gunn-Peterson Approximation. Mock LAEs created by assigning Ly$alpha$ equivalent width ($EW_text{Ly$alpha$}^text{obs}$) values to O3Es with an empirical relation also show similar, anisotropic CCFs if we use only objects with higher $EW_text{Ly$alpha$}^text{obs}$ than a certain threshold. These results indicate that galaxies on the far side of a dense region are more difficult to be detected (hidden) in Ly$alpha$ because Ly$alpha$ emission toward us is absorbed by dense neutral hydrogen. If the same region is viewed from a different direction, a different set of LAEs will be selected as if galaxies are playing hide-and-seek using HI gas. Care is needed when using LAEs to search for overdensities.
Adversarial evaluation stress tests a models understanding of natural language. While past approaches expose superficial patterns, the resulting adversarial examples are limited in complexity and diversity. We propose human-in-the-loop adversarial generation, where human authors are guided to break models. We aid the authors with interpretations of model predictions through an interactive user interface. We apply this generation framework to a question answering task called Quizbowl, where trivia enthusiasts craft adversarial questions. The resulting questions are validated via live human--computer matches: although the questions appear ordinary to humans, they systematically stump neural and information retrieval models. The adversarial questions cover diverse phenomena from multi-hop reasoning to entity type distractors, exposing open challenges in robust question answering.
Recently, the object detection based on deep learning has proven to be vulnerable to adversarial patch attacks. The attackers holding a specially crafted patch can hide themselves from the state-of-the-art person detectors, e.g., YOLO, even in the physical world. This kind of attack can bring serious security threats, such as escaping from surveillance cameras. In this paper, we deeply explore the detection problems about the adversarial patch attacks to the object detection. First, we identify a leverageable signature of existing adversarial patches from the point of the visualization explanation. A fast signature-based defense method is proposed and demonstrated to be effective. Second, we design an improved patch generation algorithm to reveal the risk that the signature-based way may be bypassed by the techniques emerging in the future. The newly generated adversarial patches can successfully evade the proposed signature-based defense. Finally, we present a novel signature-independent detection method based on the internal content semantics consistency rather than any attack-specific prior knowledge. The fundamental intuition is that the adversarial object can appear locally but disappear globally in an input image. The experiments demonstrate that the signature-independent method can effectively detect the existing and improved attacks. It has also proven to be a general method by detecting unforeseen and even other types of attacks without any attack-specific prior knowledge. The two proposed detection methods can be adopted in different scenarios, and we believe that combining them can offer a comprehensive protection.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
