Shortlisting is the task of reducing a long list of alternatives to a (smaller) set of best or most suitable alternatives from which a final winner will be chosen. Shortlisting is often used in the nomination process of awards or in recommender systems to display featured objects. In this paper, we analyze shortlisting methods that are based on approval data, a common type of preferences. Furthermore, we assume that the size of the shortlist, i.e., the number of best or most suitable alternatives, is not fixed but determined by the shortlisting method. We axiomatically analyze established and new shortlisting methods and complement this analysis with an experimental evaluation based on biased voters and noisy quality estimates. Our results lead to recommendations which shortlisting methods to use, depending on the desired properties.
Download