The digital services economy consists of online platforms that facilitate interactions between service providers and consumers. This ecosystem is characterized by short-term, often one-off, transactions between parties that have no prior familiarity. To establish trust among users, platforms employ rating systems which allow users to report on the quality of their previous interactions. However, while arguably crucial for these platforms to function, rating systems can perpetuate negative biases against marginalised groups. This paper investigates how to design platforms around biased reputation systems, reducing discrimination while maintaining incentives for all service providers to offer high quality service for users. We introduce an evolutionary game theoretical model to study how digital platforms can perpetuate or counteract rating-based discrimination. We focus on the platforms' decisions to promote service providers who have high reputations or who belong to a specific protected group. Our results demonstrate a fundamental trade-off between user experience and fairness: promoting highly-rated providers benefits users, but lowers the demand for marginalised providers against which the ratings are biased. Our results also provide evidence that intervening by tuning the demographics of the search results is a highly effective way of reducing unfairness while minimally impacting users. Furthermore, we show that even when precise measurements on the level of rating bias affecting marginalised service providers is unavailable, there is still potential to improve upon a recommender system which ignores protected characteristics. Altogether, our model highlights the benefits of proactive anti-discrimination design in systems where ratings are used to promote cooperative behaviour.
Figures & tables
Figure 1. We model interactions on digital service economy platforms with four distinct steps. First, the platform recommends a list of k providers to a user, kG of which are rated Good ; out of these, kM are part of a marginalised community. The user then chooses randomly from the list, giving those with a Good rating unit weight, and a Bad rating weight 1−γ . The chosen provider acts according to their strategy, either high or low - effort , rewarding the provider b−c or b utility respectively. The user reports the action taken to the platform, who updates the rating of the provider. User reports are biased: with rate ϵ , they will report a marginalised user who played High as Bad even when they should be rated Good .
Figure 2. We demonstrate how changing platform design affects strategy dynamics for providers. The proportion of time the dynamics spend at each state under the stationary distribution is plotted as a greyscale heatmap, and the average direction of the dynamics are overlaid as a stream plot. All subplots have k=10 and ϵ=0.3 . For subplot A , we set kG=γ=0 , which means that ϵ could take any value without affecting the dynamics. Subplot B and C have kG=5 and kG=10=k respectively. We observe that the platform’s design choices impact the providers’ strategic dynamics, leading them to adopt low-effort strategies (A), exacerbating biases by eliciting high-effort only from the dominant group (B) or inspiring high-effort by all groups (C).
Figure 3. In this Figure we fix the user population to have rating bias ϵ=0.15 and rating sensitivity γ=0.8 , and vary kG while holding k=20 . We can partition the x-axis into three distinct regions defined by whether no groups, only the dominant group, or both groups are “mostly cooperative”. We indicate the maximum value attained by each of the three evaluation metrics defined in Section 4.1 using a dashed line.
Figure 4. For k=20 (as in Figure 3 ), we vary ϵ and γ , calculating the value of kG that maximises the demographic parity ratio ( DPR ) while leaving the dynamics in regime C .
Figure 5. Consider a platform with k=20 and vary ϵ and γ between low and high values. For each resulting scenario, find the value of kG that maximises UX×DPR subject to kM=0 , we plot this as a faint grey vertical line. As we now vary kM , the dynamics stay inside regime C and the evaluation metrics peak at the same value of kM when γ is high enough (0.8).
Figure 6. Let k = 20 , ϵ=0.35 , and γ=0.65 , and vary the uncertainty of rating bias. As previously established: forcing kM=0 leads to a low DPR . Even when uncertainty is 0.7, allowing kM≥0 still results in a better worst-case outcome. Optimisation of max-min DPR and average DPR are both fine choices of metric to optimise over that only diverge significantly in outcome for very large uncertainty.