Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userDesigning Informative Rating Systems: Evidence from an Online Labor Market
46
0
0.0
(
0
)
Added by
Nikhil Garg
Publication date
2018
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
Platforms critically rely on rating systems to learn the quality of market participants. In practice, however, these ratings are often highly inflated, and therefore not very informative. In this paper, we first investigate whether the platform can obtain less inflated, more informative ratings by altering the meaning and relative importance of the levels in the rating system. Second, we seek a principled approach for the platform to make these choices in the design of the rating system. First, we analyze the results of a randomized controlled trial on an online labor market in which an additional question was added to the feedback form. Between treatment conditions, we vary the question phrasing and answer choices; in particular, the treatment conditions include several positive-skewed verbal rating scales with descriptive phrases or adjectives providing specific interpretation for each rating level. The online labor market test reveals that current inflationary norms can in fact be countered by re-anchoring the meaning of the levels of the rating system. In particular, the positive-skewed verbal rating scales yield rating distributions that significantly reduce rating inflation and are much more informative about seller quality. Second, we develop a model-based framework to compare and select among rating system designs, and apply this framework to the data obtained from the online labor market test. Our simulations demonstrate that our model-based framework for scale design and optimization can identify the most informative rating system and substantially improve the quality of information obtained over baseline designs. Overall, our study illustrates that rating systems that are informative in practice can be designed, and demonstrates how to design them in a principled manner.
rate research
Read More
Modern online platforms rely on effective rating systems to learn about items. We consider the optimal design of rating systems that collect binary feedback after transactions. We make three contributions. First, we formalize the performance of a rating system as the speed with which it recovers the true underlying ranking on items (in a large deviations sense), accounting for both items underlying match rates and the platforms preferences. Second, we provide an efficient algorithm to compute the binary feedback system that yields the highest such performance. Finally, we show how this theoretical perspective can be used to empirically design an implementable, approximately optimal rating system, and validate our approach using real-world experimental data collected on Amazon Mechanical Turk.
Lack of skills is arguably one of the most important determinants of high levels of unemployment and poverty. In response, policymakers often initiate vocational training programs in effort to enhance skill formation among the youth. Using a regression-discontinuity design, we examine a large youth training intervention in Nepal. We find, twelve months after the start of the training program, that the intervention generated an increase in non-farm employment of 10 percentage points (ITT estimates) and up to 31 percentage points for program compliers (LATE estimates). We also detect sizeable gains in monthly earnings. Women who start self-employment activities inside their homes largely drive these impacts. We argue that low baseline educational levels and non-farm employment levels and Nepals social and cultural norms towards women drive our large program impacts. Our results suggest that the program enables otherwise underemployed women to earn an income while staying at home - close to household errands and in line with the socio-cultural norms that prevent them from taking up employment outside the house.
Although freelancing work has grown substantially in recent years, in part facilitated by a number of online labor marketplaces, (e.g., Guru, Freelancer, Amazon Mechanical Turk), traditional forms of in-sourcing work continue being the dominant form of employment. This means that, at least for the time being, freelancing and salaried employment will continue to co-exist. In this paper, we provide algorithms for outsourcing and hiring workers in a general setting, where workers form a team and contribute different skills to perform a task. We call this model team formation with outsourcing. In our model, tasks arrive in an online fashion: neither the number nor the composition of the tasks is known a-priori. At any point in time, there is a team of hired workers who receive a fixed salary independently of the work they perform. This team is dynamic: new members can be hired and existing members can be fired, at some cost. Additionally, some parts of the arriving tasks can be outsourced and thus completed by non-team members, at a premium. Our contribution is an efficient online cost-minimizing algorithm for hiring and firing team members and outsourcing tasks. We present theoretical bounds obtained using a primal-dual scheme proving that our algorithms have a logarithmic competitive approximation ratio. We complement these results with experiments using semi-synthetic datasets based on actual task requirements and worker skills from three large online labor marketplaces.
This paper is to analyze the properties of evolving bipartite networks from four aspects, the growth of networks, the degree distribution, the popularity of objects and the diversity of user behaviours, leading a deep understanding on the empirical data. By empirical studies of data from the online bookstore Amazon and a question and answer site Stack Overflow, which are both rating bipartite networks, we could reveal the rules for the evolution of bipartite networks. These rules have significant meanings in practice for maintaining the operation of real systems and preparing for their future development. We find that the degree distribution of users follows a power law with an exponential cutoff. Also, according to the evolution of popularity for objects, we find that the large-degree objects tend to receive more new ratings than expected depending on their current degrees while the small-degree objects receive less ratings in terms of their degrees. Moreover, the user behaviours show such a trend that the larger degree the users have, the stronger purposes are with their behaviours except the initial periods when users choose a diversity of products to learn about what they want. Finally, we conclude with a discussion on how the bipartite network evolves, which provides guideline for meeting challenges brought by the growth of network.
Labor market institutions are central for modern economies, and their polices can directly affect unemployment rates and economic growth. At the individual level, unemployment often has a detrimental impact on peoples well-being and health. At the national level, high employment is one of the central goals of any economic policy, due to its close association with national prosperity. The main goal of this thesis is to highlight the need for frameworks that take into account the complex structure of labor market interactions. In particular, we explore the benefits of leveraging tools from computational social science, network science, and data-driven theories to measure the flow of opportunities and information in the context of the labor market. First, we investigate our key hypothesis, which is that opportunity/information flow through weak ties, and this is a key determinant of the length of unemployment. We then extend the idea of opportunity/information flow to clusters of other economic activities, where we expect the flow within clusters of related activities to be higher than within isolated activities. This captures the intuition that within related activities there are more capitals involved and that such activities require similar capabilities. Therefore, more extensive clusters of economic activities should generate greater growth through exploiting the greater flow of opportunities and information. We quantify the opportunity/information flow using a complexity measure of two economic activities (i.e. jobs and exports).
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
