Search engines such as Google or Bing offer results that present matches with the literal meaning of the search carried out, in combination with factors managed by their own algorithm.
In order to develop an option that provides more precise results, providing exposure even to those who, despite being related to the search terms, are not among the first, researchers at Cornell University developed a fairer classification system for searches that can range from hotels to job offers and audiovisual content.
New alternative to offer better results in search engines
The new Ranking system introduced by the Cornell team takes care of delivering relevant results, but divides the user’s attention more evenly among the search results. It can be applied to online marketplaces such as travel sites, recruitment platforms, and news aggregators.
The systems that manage the results returned by a search engine are generally responsible for classifying the content to be displayed according to what users want to see. In practice, however, many items are unfairly ranked low. Under this dynamic, sites of similar content and quality can end up far apart in the rankings, in some cases accessing almost zero exposure.
“In recommendation systems and search engines, whoever ranks high gets a big benefit from that”, said Thorsten Joachims, professor of computer science and information science at Cornell. “The user’s attention is a limited resource and we need to distribute it equally among the elements”he added.
To remedy this situation, Yuta Saito, a Ph.D. student in computer science at Cornell, developed an improved classification system, drawing on ideas from economic theory. For this, he applied the principle of «fair division«, applied to distribute limited resources in the fairest way possible among the members of a group.
The fair division of results is defined based on three criteria: each item’s benefit of being ranked on the platform is better than being discovered randomly; the impact of no element, such as income, can be easily improved; and no item would gain an advantage by changing its ranking compared to other items in a series of searches.
The implementation of this principle was tested with a sample of synthetic data and another of real-world data, effectively demonstrating the viability of the proposed system.
“We completely redefined fairness in qualifying”Saito said. “It can be applied to any type of two-sided classification system”he added.
Illustrating the impact of this system, the team used its application on YouTube as an example. In this case, the recommendation system would present a more varied stream of videos, which could distribute earnings more evenly among content creators. “We want to satisfy the users of the platform, of course, but we must also be fair to the video creators, to maintain their diversity in the long term”Saito said.
In other areas of use, such as job platforms, this system could diversify the search results with more precise results, instead of always showing the featured or most popular offers that are shown by default. Similarly, extrapolating this technique to other areas, it could also be used to improve movie recommendations on streaming platforms, facilitate the search for relevant content at conferences, and even access a more balanced selection of curated news.
The technical details of this proposal were presented at the publication “Fair Ranking as Fair Division: Impact-Based Individual Fairness in Ranking”, detached from the event “Proceedings of the 2022 Association for Computing Machinery Special Interest Group on Knowledge Discovery and Data Mining Conference”.