Popularity bias reduction in multi-criteria recommender systems: the RelPref and PriRelPref approaches

Introduction

With the rapid advancement of digital technologies and the increasing integration of online platforms into everyday life, users are now confronted with an unprecedented number of product and service options. While this expansion provides individuals with a broader pool of choices across various domains, it also renders decision-making processes more complex and time-consuming (Sarwar et al., 2001). For instance, selecting a movie that aligns with personal preferences from among thousands of titles on a streaming platform, or identifying the most suitable accommodation from hundreds of hotel options during travel planning, can impose a significant cognitive burden on users (Pazzani & Billsus, 2007). In this context, recommender systems have emerged as critical tools in digital interaction environments, designed to help users manage the overwhelming abundance of information and facilitate decision-making processes (Burke, 2002).

Recommender systems aim to predict items or content that users are likely to prefer by analyzing their historical interactions or explicitly stated preferences. Traditional approaches generally rely on a single overall rating to measure item quality and to identify similarities between users or products (Adomavicius & Kwon, 2012). While such methods can be effective in certain cases, they fail to capture the inherently multidimensional structure of items and the heterogeneity of user preferences (Lakiotaki, Matsatsinis & Tsoukias, 2011). For instance, in movie recommendation scenarios, one user may emphasize the importance of story and acting, whereas another may value visual effects or the director’s style. Similarly, in hotel recommendations, some users may prioritize location, while others focus more on cleanliness, customer service, or available amenities (Sahoo, Singh & Mukhopadhyay, 2012). To address this diversity, Multi-Criteria Recommender Systems (MCRS) have been developed, enabling users to evaluate items along multiple dimensions rather than providing only a single overall score (Chen, Chen & Wang, 2015). By analyzing ratings for each criterion separately, MCRS are capable of generating more comprehensive and personalized recommendations, highlighting items that may not have the highest overall rating but perform strongly in the criteria that matter most to a specific user (Lee et al., 2016). However, the effectiveness of such systems largely depends on accurately interpreting user rating behaviors. Identical scores may convey different meanings depending on individual rating tendencies and contextual preferences. For example, consider two users, $u$ and $v$, who both assign a score of 5 to item $i_5$ within the set $\{i_1,i_2,i_3,i_4,i_5\}$ (Fig. 1). If user $v$ frequently gives maximum ratings, this score may reflect a generally lenient rating behavior. Conversely, if user $u$ rarely assigns five stars, the same score may indicate a particularly strong level of satisfaction. Therefore, user $u$’s preference for $i_5$ is stronger than that of user $v$, despite the identical absolute ratings assigned by both users. This observation highlights the importance of modeling relative rating behaviors and contextual tendencies to achieve genuinely personalized and reliable recommendations.

Therefore, structurally modeling users’ rating behavior is a critical factor in enhancing the personalization quality of recommendations. Relative interpretations of ratings, based on individual tendencies and rating distributions, offer a more nuanced understanding of user preferences—something that MCRS is uniquely equipped to exploit.

Although MCRS extend the capabilities of traditional single-criterion systems by capturing users’ preferences across multiple dimensions, they also introduce unique challenges. Among these, one of the most critical is popularity bias, which undermines fairness, diversity, and long-term user satisfaction (Abdollahpouri, Burke & Mobasher, 2017). Popularity bias refers to the tendency of recommendation algorithms to disproportionately favor items with high visibility, such as those with many ratings or frequent interactions. Consequently, niche or long-tail items lose exposure, reducing overall discoverability (Park & Tuzhilin, 2008). In multi-criteria contexts, this problem becomes even more complex: an item may appear favorable based on its aggregated overall score, even if it performs poorly in the specific criteria most valued by a user. This discrepancy weakens personalization quality and diminishes the fairness of recommendations.

Recent studies such as Dynamic User-oriented re-Ranking (DUoR), Popularity-aware item weighting (Paiw), and other popularity-debiasing models have primarily focused on single-criterion recommendation settings, aiming to balance accuracy and diversity or to mitigate popularity bias through re-ranking and calibration strategies (Gulsoy et al., 2025). However, these conventional approaches overlook the inherently multi-dimensional nature of user evaluations in MCRS, where popularity bias may arise and propagate differently across distinct criteria. In this context, the present study focuses on evaluating classical re-ranking methods (DUoR, Paiw, Augmentative (Aug), Multiplicative (Mul)) that are specifically designed to alleviate popularity bias. These methods provide an interpretable and methodologically consistent comparison framework, which enables a transparent and systematic assessment of bias mitigation performance. Although recent neural approaches, such as neural graph collaborative filtering and counterfactual debiasing, have demonstrated promising potential for addressing bias and fairness issues in recommendation tasks (Wang et al., 2019; Wei et al., 2021), the present work deliberately excludes deep learning–based recommenders from its experimental scope. This design choice aims to maintain analytical clarity and isolate the effects of re-ranking strategies without introducing additional representational or optimization complexities that could confound the comparative analysis.

Building on this limitation, despite the increasing attention toward fairness and bias mitigation in recommender systems, there remains a notable gap in understanding how popularity bias manifests, interacts, and propagates within multi-criteria structures. Addressing this research gap, the current study presents the first systematic analysis of popularity bias in MCRS and proposes a novel framework that incorporates users’ criterion-level priorities into the recommendation process. By aligning item strengths with the aspects most valued by individual users, the proposed approach effectively reduces the dominance of overly popular items and promotes recommendation lists that are balanced, diverse, and fairness-aware.

The main contributions of this study are as follows:

• A baseline model tailored to multi-criteria recommender systems is proposed to address popularity bias. This foundational approach offers a systematic reference point for objectively evaluating the performance of new methods designed to mitigate the bias.

• A novel method is developed to reduce popularity bias in next-generation multi-criteria recommender systems. Unlike traditional approaches, this method considers the alignment between users’ most valued criteria and the dimensions in which items excel, fostering a more user-sensitive and impartial recommendation mechanism.

• To better analyze the effects of popularity bias in multi-criteria settings, three distinct scenarios centered on user rating behavior are proposed. These scenarios allow for a detailed examination of users’ rating tendencies, scale perceptions, and criteria-based sensitivities, thus enhancing the integration of rating dynamics into the recommendation model and improving personalization accuracy.

• The impact of the proposed method on mitigating popularity bias is comprehensively evaluated using multi-dimensional metrics such as long-tail item visibility and diversity (Gini index). This evaluation not only assesses user-level benefits but also illustrates the method’s implications for system-level fairness and inclusivity.

• This study is the first to systematically address popularity bias within the context of multi-criteria recommender systems. By filling a critical gap in the literature, it lays an important theoretical and methodological foundation for future research in this domain.

The remaining parts of our article are organized as follows: ‘Preliminaries’ provides a comprehensive review of the literature on recommender systems and popularity bias. ‘Proposed Approaches for Popularity Bias Mitigation in Multi-Criteria Recommender Systems’ details the methods and algorithms employed in the study. The proposed methodologies and algorithms are introduced in ‘Experiments’. ‘Experimental Results’ describes the experimental design and the performance metrics utilized for evaluation. The results of the experimental analysis are presented in ‘Discussion’. Finally, ‘Conclusion and Future Work’ presents the discussion of the findings, followed by the conclusions and directions for future research.

Related work

The evolution of recommendation systems and the problem of popularity bias has become a significant research area attracting attention from researchers across different disciplines in recent years. When examining the studies in the literature, a clear evolutionary process can be observed, ranging from the fundamental building blocks of recommendation systems to today’s complex multi-criteria approaches. Throughout this evolution, it is evident how both technical and user-oriented aspects of the systems have developed, and how the popularity bias problem has been addressed from different dimensions.

In early studies, introduced the collaborative filtering approach, laying the foundation for recommendation systems by computing user similarities based on single rating values. This idea was later extended by Sarwar et al. (2001) through matrix factorization techniques, enabling more efficient handling of user-item interactions. Similarly, Pazzani & Billsus (2007) developed content-based filtering approaches that generated user profiles using single-criterion evaluations, while (Burke, 2002) proposed hybrid recommendation models to enhance diversity and mitigate the limitations of pure collaborative or content-based systems. The introduction of latent factor models by Koren, Bell & Volinsky (2009) further improved the modeling of user-item interactions, and (Su & Khoshgoftaar, 2009) provided a comprehensive overview of similarity-based and content-driven prediction methods for single-criterion systems. However, the major limitation of these studies was one-dimensional treatment of user preferences. This limitation motivated the emergence of MCRS, which allow users to evaluate items across multiple aspects. Adomavicius & Kwon (2012) conducted one of the pioneering studies highlighting the significance of incorporating multiple criteria, while Lakiotaki, Matsatsinis & Tsoukias (2011) analyzed detailed criterion-weighting mechanisms. Sahoo, Singh & Mukhopadhyay (2012) explored the influence of different criteria in movie recommendations, and (Chen, Chen & Wang, 2015) demonstrated the importance of multi-dimensional evaluation in hotel recommendation systems. Collectively, these studies revealed the necessity of modeling the multi-faceted nature of user preferences and set the foundation for modern MCRS research.

Building on this foundation, recent studies have aimed to improve the accuracy, diversity, and robustness of multi-criteria recommender systems through hybrid and deep learning-based models. Batmaz & Kaleli (2019) proposed an autoencoder-based MCRS (AE-MCCF) that effectively captured nonlinear user–item relationships, while (Batmaz & Kaleli, 2024) extended this approach with a hybrid similarity-based model integrating latent features from reviews and criteria ratings to alleviate sparsity. In addition, Kaya & Kaleli (2022) introduced an intuitionistic fuzzy set-based top- $n$ method enhancing diversity and novelty, and (Kaya & Kaleli, 2023) analyzed the robustness of multi-criteria top- $n$ algorithms against shilling attacks. To address such vulnerabilities, Kaya, Yalcin & Kaleli (2023) proposed a classification-based detection model that distinguishes genuine and malicious profiles using behavioral and popularity-based features. Furthermore, Adomavicius & Kwon (2012) demonstrated that ranking-based post-processing techniques can enhance aggregate diversity without compromising accuracy, inspiring subsequent research to balance personalization, diversity, and robustness in multi-criteria environments.

In addition to the detailed discussion above, Table 1 provides a summary of key studies in the field of multi-criteria recommender systems, highlighting methodological contributions and comparative performance characteristics.

Following these advancements in accuracy and robustness, another critical line of research has focused on addressing popularity bias, a pervasive issue observed in both single-criterion and multi-criteria recommender systems. Popularity bias arises when recommendation algorithms disproportionately favor frequently rated or well-known items, thereby reducing exposure to less popular but potentially relevant alternatives. Recent studies in this area, summarized in Table 2, provide a methodological synthesis of prior studies on popularity bias in recommender systems. Existing research mainly focuses on analyzing popularity dominance, improving calibration, or enhancing long-tail exposure. However, most approaches address popularity bias within single-criterion settings and do not explicitly model how bias propagates across multiple evaluation dimensions. Moreover, user-specific criterion priorities and multi-criteria preference structures are largely overlooked, highlighting a critical research gap addressed by the proposed framework.

Overall, the studies summarized in the related tables reveal a clear methodological separation in the existing literature. On the one hand, a substantial body of work investigates multi-criteria recommender systems, yet these approaches largely overlook the presence and impact of popularity bias within multi-dimensional preference structures. On the other hand, studies that explicitly address popularity bias and fairness concerns predominantly focus on single-criterion recommendation settings, where user preferences are modeled along a single evaluation dimension. As a result, the interaction between popularity bias and multi-criteria preference modeling remains insufficiently explored.

The present study bridges this gap by jointly considering popularity bias mitigation and multi-criteria recommendation modeling within a unified framework. By integrating criterion-level user preferences into the debiasing process, the proposed approach combines the strengths of both research streams and addresses their respective limitations. This integration enables a more realistic and comprehensive treatment of popularity bias as it manifests across multiple evaluation dimensions, thereby providing a novel contribution to the literature on fair and personalized recommender systems.

Preliminaries

In this section, necessary information about the methods used in the study is given.

Proposed approaches for popularity bias mitigation in multi-criteria recommender systems

In this study, two novel algorithms are proposed to mitigate popularity bias in recommender systems: the Relative Preference Model (RelPref) and the Criterion-Aware Relative Preference Model (PriRelPref). These algorithms are designed to overcome the limitations of existing approaches and provide a more balanced distribution of content in recommendation lists. Also, RelPref and PriRelPref are first approaches that performed on multi-criteria systems.

Relative preference model

The RelPref is a dynamic and user-centric re-ranking approach designed to mitigate the widespread popularity bias commonly observed in recommender systems. The system architecture, illustrated in Fig. 2, outlines the operational workflow of the algorithm on multi-criteria datasets. This algorithm focuses on understanding users’ tendencies toward item popularity and aims to generate fairer and more personalized recommendations by balancing these tendencies throughout the recommendation process.

The proposed RelPref framework is structured around two main components: the Multi-Criteria Dataset and the re-ranking module called RelPref. The first component comprises three distinct types of data representations: overall-based, criteria-based, and aggregate-based. These representations allow for a multidimensional analysis of user rating behaviors, enabling the estimation of popularity at the level of individual criteria (e.g., story, acting, directing, visuals), which are then integrated into the decision-making process of the algorithm. The core component, RelPref, is responsible for the user-focused re-ranking procedure. The process begins by generating a prediction matrix $P_{1\times n}$ for the test user u using a standard Collaborative Filtering model. This matrix contains the predicted scores for items that the user has not yet rated but may find relevant. The predicted scores are sorted to form a candidate item set $C_{1\times m}$. Items in the candidate set are then categorized into head (popular) and tail (less popular) groups based on relative item popularity levels. Subsequently, the user’s individual popularity tendency, referred to as the popularity ratio ( $PopRati{o}_u$), is computed (see ‘User Popularity Bias Metrics in Recommender Systems’). This ratio serves as a personalized metric indicating the user’s historical preference toward popular items. The item with the highest predicted score is selected from the candidate set and added to the recommendation list ( $RecList$). During this selection, the popularity degree of the current recommendation list ( $Pre{f}_{RecList}$) is calculated based on the category of the selected item.

At this stage, item selection is no longer governed solely by predicted scores but also incorporates a metric reflecting the user’s relative preference tendencies. For each candidate item, a user-specific preference score $pref(r_{ui})$ is computed. This score is inspired by the Borda count method and accounts for the item’s relative position within the user’s rating history. Specifically, for each rated item i with rating $r_{ui}$, the following are calculated:

• •

  The number of items rated lower: $$pre{f}^{-}(r_{ui})=\left|\{j\in R_u^{+}\mid r_{uj}<r_{ui}\}\right|.$$

• •

  The number of items rated equally: $$pre{f}^{=}(r_{ui})=\left|\{j\in R_u^{+}\mid r_{uj}=r_{ui}\}\right|.$$

• •

  These components are linearly combined to obtain the final relative preference score: $$pref(r_{ui})=\alpha \cdot pre{f}^{-}(r_{ui})+\beta \cdot pre{f}^{=}(r_{ui})$$where $\alpha$ and $\beta$ are weighting parameters, typically set to $\alpha =1$ and $\beta =0.5$ in the literature. This method allows for more nuanced ranking distinctions among equally rated items, reflecting the user’s overall rating distribution.

The computed $Pre{f}_{RecList}$ is compared against the user’s $PopRati{o}_u$. If the list’s popularity level is lower than the threshold, the next item is selected from the candidate head category; otherwise, from the candidate tail category. This iterative procedure continues until the recommendation list reaches the desired length ( $topN$). Consequently, the RelPref module balances both predicted relevance and inferred user preferences, leading to more personalized and diversity-aware recommendations. One of the most distinctive features of the algorithm is its ability to dynamically shift between popularity categories based on individual user tendencies. The RelPref framework continuously monitors the popularity balance of the recommendation list at each step, thereby maintaining diversity while enhancing personalization. Compared to the classical DUoR algorithm, this framework presents a significant advancement. By leveraging criterion-specific popularity analysis, it enables detailed examination of user behavior patterns across different aspects, which are then seamlessly incorporated into the recommendation logic. The RelPref algorithm offers notable advantages in mitigating popularity bias, increasing recommendation diversity, and enhancing user satisfaction, particularly in multi-criteria datasets. As illustrated in Fig. 2, the modular and adaptive architecture of the framework clearly demonstrates how recommendation generation can be both preference-aware and popularity-sensitive.

Table 3 outlines the core steps of the RelPref algorithm along with the methods employed at each stage. The process begins with the representation of multi-criteria user feedback, supporting various data types including overall-based, criteria-based, and aggregate-based formats. Subsequently, items are classified into head and tail categories based on the Pareto principle and global popularity measures. The collaborative filtering is then used to generate predicted scores, from which a candidate set of items is formed. The user’s popularity tendency is computed using the RPI, APRI, and BTA-extended variants, which informs the selection of items for the recommendation list. If necessary, the list is re-ranked according to the user’s preferences and popularity orientation. Finally, a fixed-length recommendation list is produced. This structured approach aims to enhance personalization and diversity by accounting for individual user tendencies toward popular or less-known items.

Table 3:

Overview of the RelPref algorithm: steps and methodological components.

Step no	Step description	Method/Process used
1	Multi-Criteria Data Input and Representation	Data types: Overall-based, Criteria-based, Aggregate-based
2	Separation of Head/Tail Items	Pareto principle, classification based on global popularity
3	Predicted Rating Matrix ( $P_{1\times n}$)	Collaborative Filtering
3.1	Generation of Candidate Item Set ( $C_{1\times m}$)	Sorting by predicted scores, selection of top 1 items
3.2	Construction of Recommendation List ( $RecList$)	Item selection from candidate head/tail based on user tendency, computation of $Pre{f}_{RecList}$
3.3	Re-ranking Within the Recommendation List	Item selection based on comparison of $Pre{f}_{RecList}$ and $PopRati{o}_u$
4	Calculation of User Popularity Tendency ( $PopRati{o}_u$)	RPI, APRI, $BT{A}_{RPI}$, $BT{A}_{APRI}$ methods
5	Termination of the Process	Creation of final recommendation list with fixed length

DOI: 10.7717/peerj-cs.3658/table-3

Criterion-aware relative preference modell

The Criterion-Aware Relative Preference Model (PriRelPref) is an advanced re-ranking framework designed to enhance the fairness and personalization of recommendations by explicitly incorporating user-specific dominant criteria into the recommendation logic. As an extension of the RelPref, the PriRelPref framework preserves the popularity-aware and diversity-sensitive characteristics of its predecessor while refining personalization through multi-criteria learning. The architecture of the proposed PriRelPref model is depicted in Fig. 3.

The framework consists of two key modules: (i) the Dominant Criterion Identification Layer, and (ii) the Criterion-Aware RelPref Engine.

Dominant Criterion Identification: The first module introduces an essential personalization enhancement by determining the dominant evaluation dimension (criterion) for each user in a multi-criteria rating dataset. This is accomplished through a Multiple Linear Regression (MLR) model trained to predict the user’s overall rating as a function of multiple sub-criteria (e.g., story, acting, directing, visuals). For each user $u$, the following linear model is employed:

(5) $${\hat{r}}_u={\gamma }_1\cdot c_{1u}+{\gamma }_2\cdot c_{2u}+\cdots +{\gamma }_k\cdot c_{ku}+\epsilon$$where:

• ${\hat{r}}_u$ is the predicted overall rating for user $u$,

• $c_{iu}$ is the rating of user $u$ on the $i$-th criterion,

• ${\gamma }_i$ is the learned weight for criterion $i$,

• $\epsilon$ is the residual error.

The criterion with the highest regression coefficient ${\gamma }_i$ is designated as the dominant criterion for that user. This dominant criterion reflects the most influential aspect of user evaluation and guides the subsequent re-ranking stage.

Criterion-Aware RelPref Engine: Once the dominant criterion for user $u$ is identified, the re-ranking phase proceeds similarly to the RelPref model, but with all popularity analysis and preference calculations based on the dominant criterion.

By embedding user-specific criterion importance into the re-ranking logic, PriRelPref introduces a nuanced personalization approach that significantly enhances both the relevance and diversity of recommendations. This makes the model particularly effective for domains such as movies, hotels, or multi-attribute products, where user evaluations span multiple dimensions.

Experiments

In this section, a set of experiments are conducted to evaluate baseline, RelPref, and PriRelPref methods. Firstly, the experimental methodology, dataset, and performance evaluation measures are briefly mentioned.

Experimental results

In this study, we investigate the growing challenge of popularity bias in recommender systems, particularly within multi-criteria rating environments. Our primary objective is to introduce two novel algorithms, RelPref and PriRelPref, which aim to mitigate popularity bias by modeling contextual variations in user rating behavior. These algorithms capture user-specific tendencies and interpret rating patterns relative to individual behavioral contexts. For comparison, four established baseline methods—DUoR, Paiw, Mul, and Aug—are also included in the evaluation. The experiments are conducted on three multi-criteria datasets: YM20, YM10, and TA. The evaluation framework is designed to assess both recommendation accuracy and the extent to which each method mitigates popularity bias. To estimate users’ inherent inclination toward popular items, four distinct popularity tendency metrics are employed: APRI, RPI, $BT{A}_{\mathrm{A}\mathrm{P}\mathrm{R}\mathrm{I}}$, and $BT{A}_{\mathrm{R}\mathrm{P}\mathrm{I}}$. Each recommendation algorithm is evaluated under all four estimation strategies, enabling a detailed and nuanced analysis of performance across different user profiles. A comprehensive set of evaluation metrics is used to assess the recommendation outcomes, including Precision, Recall, F1-measure, Novelty, APLT, LTC, and the Gini index. The experiments follow a standard leave-one-out protocol in which one user is designated as the test subject and the remaining users constitute the training set. For each test instance, popularity tendency scores are computed using the four estimation methods, after which the recommendation algorithms generate top-N lists conditioned on these scores. This experimental design provides a systematic and equitable basis for comparing the proposed models with the baseline approaches, particularly regarding ability to balance accuracy and diversity while mitigating popularity bias.

The experimental results for the TA dataset are presented in Figs. 4–7. For the first popularity estimation method, $BT{A}_{RPI}$, three different analytical perspectives are examined. The criteria-based analysis (Fig. 4B) evaluates user assessments of location, cleanliness, service quality, value-for-money, and room quality individually. The overall-based analysis (Fig. 4C) considers general user evaluations, while the aggregate-based analysis (Fig. 4A) derives results by combining all criteria through weighted averaging.

Overall and criteria-based scenarios show that RelPref demonstrates substantially higher percentage improvements in the Precision metric compared to the other methods (see Figs. 4C, 4B). In particular, RelPref achieves approximately a 130% improvement over Paiw, and this ratio increases even further when compared to Mul. Although the proportion of popular items in RelPref’s recommendation lists remains relatively high, this does not undermine performance; instead, it establishes a balanced structure between accuracy and diversity, ultimately enhancing user satisfaction. In contrast, methods such as Paiw, which excessively promote non-popular items, lead to significant declines in user satisfaction. This finding clearly indicates that reducing popularity bias alone is not sufficient to ensure effective recommendations. From the perspective of the Gini index, RelPref exhibits approximately 96% higher Gini values than Mul, highlighting its substantially more diverse recommendation distribution across these two scenarios. PriRelPref, on the other hand, presents a more fluctuating pattern in terms of Precision, outperforming some methods while falling behind others. However, when examined within the aggregate-based scenario, PriRelPref emerges as the superior approach. Although PriRelPref performs weaker in Precision relative to the other scenarios and methods, it demonstrates a stronger performance in Novelty and APLT. The approximately 61.86% improvement observed in this scenario indicates that PriRelPref achieves a notable advantage in terms of diversity. Moreover, the reduction of the Gini Index value to 0.3327 demonstrates that PriRelPref attains the most balanced recommendation distribution among all methods evaluated.

Another user popularity bias approach, the $BT{A}_{APRI}$ method, was also evaluated on the same dataset (see Fig. 5). The results revealed that this method exhibited trends largely similar to those of the $BT{A}_{RPI}$ method. In terms of accuracy, there was no notable difference between the two approaches (for example, Precision $<0.0065$, i.e., approximately 0.65%), indicating that both models behave consistently in capturing user preferences and generating recommendations. Therefore, the interpretations made for the $BT{A}_{RPI}$ method can also be extended to $BT{A}_{APRI}$. However, when examining the Novelty ratios, it is observed that methods such as Mul, Paiw, and Aug achieved significantly high values, exceeding 90%. This finding indicates that these algorithms tend to recommend items that users have not previously encountered, effectively mitigating popularity bias. Similar trends were also observed in the proportion of non-popular items within the recommendation lists. Nevertheless, it is important to note that a higher novelty or lower popularity ratio does not necessarily correspond to greater user satisfaction. Despite the presence of a large number of non-popular items in the recommendation lists, users were often dissatisfied with the generated recommendations, suggesting that these models fail to align with actual user preferences. In this context, the proposed RelPref method establishes a more balanced trade-off between accuracy and debiasing performance. RelPref succeeds in maintaining user satisfaction while reducing the dominance of popular items. The method increases diversity and fairness in recommendations while still preserving alignment with user interests. However, due to the high sparsity of the TA dataset, the proportion of popular items recommended by RelPref remained relatively low. This limitation stems from the limited number of user–item interactions available in the dataset, which can indirectly influence the overall recommendation performance.

In summary, the comparison between the $BT{A}_{APRI}$ method and other baseline approaches highlights that merely reducing popularity bias is insufficient. Instead, popularity mitigation should be harmonized with accuracy, diversity, and user satisfaction to achieve a fair and effective recommendation process. From this perspective, RelPref demonstrates a more stable and sustainable approach by preserving user satisfaction while effectively alleviating popularity bias.

The results of the APRI method indicate that, as presented in Fig. 6, the performance rankings of certain algorithms differ compared to previous methods across both criteria-based (Fig. 6B) and aggregate-based (Fig. 6A) scenarios. For instance, when examining the precision metric of the DuoR algorithm, the APRI method leads to an approximate 26.15% decrease, demonstrating a notable decline in accuracy. In contrast, an analysis of the RelPref method under the same scenarios reveals no substantial change in system accuracy; however, significant improvements are observed in terms of diversity, novelty, and the proportion of less popular items appearing in recommendation lists. These findings suggest that, within the APRI framework, the RelPref method provides a more balanced and effective structure, particularly in enhancing the diversity of recommendations presented to users.

The final popularity bias mitigation approach applied to this dataset, the RPI method, reveals only minor fluctuations compared to the previously examined techniques (see Fig. 7). Although the proposed RelPref method achieves relatively higher performance in terms of accuracy and user satisfaction (approximately 0.65%), this improvement is not consistently reflected in other evaluation dimensions such as novelty and diversity. Conversely, methods that perform strongly in beyond-accuracy metrics, including novelty and diversity, generally fail to deliver comparable levels of user satisfaction. This inverse relationship highlights the inherent trade-off between reducing popularity bias and preserving recommendation relevance.

Overall, when evaluating the results obtained from the TA dataset, it becomes evident that the dataset’s structural characteristics play a crucial role in shaping these outcomes. The TA dataset exhibits a high level of sparsity, meaning that many users have provided ratings for only a limited number of items. Furthermore, the analyses indicate a relatively high correlation among the evaluation criteria, suggesting that users tend to rate multiple aspects of a hotel (such as location, cleanliness, service quality, and value-for-money) in a similar manner. This strong inter-criteria correlation reduces the distinctiveness between different criteria and ultimately leads to comparable results across the criteria-, overall-, and aggregate-based analyses.

For the YM20 multi-criteria dataset, the first analysis is conducted using the $BT{A}_{RPI}$ method (see Fig. 8). The aggregate-based results show that DUoR and RelPref achieve the highest levels of user satisfaction. On average, users are satisfied with approximately two of the ten recommended items, one of which typically belongs to the long-tail. This indicates that the proposed methods not only maintain accuracy but also promote exposure to less popular items, thereby supporting diversity and discovery. In the criteria-based scenario, substantial performance differences emerge among the compared methods (can see Fig. 8B). Notably, RelPref and PriRelPref outperform the baselines, achieving more than a 20% improvement in Precision. These results demonstrate the superior ability of the proposed approaches to capture user-specific preferences under multi-criteria conditions. In contrast, Mul, Paiw, and Aug display overly aggressive debiasing behavior by recommending a large volume of niche, non-popular items, which results in a marked decline in user satisfaction. Under the $BT{A}_{RPI}$ configuration, the findings reaffirm users’ general tendency to prefer popular items. However, the proposed methods effectively sustain user satisfaction by selectively incorporating long-tail items without sacrificing accuracy. From a diversity perspective, RelPref (Gini: 0.8874) surpasses DUoR by 2.4%, reflecting its capacity to balance satisfaction with fair item exposure. Conversely, PriRelPref (Gini: 0.7651) performs 11.7% lower than DUoR, suggesting that it prioritizes user satisfaction at the expense of fairness.

Within the $BT{A}_{APRI}$ framework, the proposed RelPref and PriRelPref methods demonstrate a clear improvement in performance, as illustrated in Fig. 9. Both approaches achieve higher user satisfaction while simultaneously increasing item diversity in the recommendation lists. The results indicate that the proposed methods effectively balance accuracy and diversity, outperforming baseline approaches that rely solely on aggressive popularity mitigation. In the overall-based and criteria-based scenarios (see Figs. 9C and 9B), the proposed methods consistently generate more satisfactory recommendation lists. For a top10 recommendation list, approximately three items meet user expectations, while diversity levels noticeably increase. This outcome highlights the ability of RelPref and PriRelPref to maintain alignment with user preferences without sacrificing exposure to less-popular items. In the aggregate-based scenario, RelPref achieves the strongest overall performance (Precision: 0.1431, Recall: 0.0475, F1-score: 0.064), whereas baseline methods such as Aug, Mul, and Paiw exhibit consistently low effectiveness (see Fig. 9A). Across all analyses, these baseline approaches also produce the highest Gini values, indicating lower fairness in item exposure. By contrast, the proposed methods adopt a more balanced strategy, preserving moderate fairness while delivering higher user satisfaction. Overall, the findings suggest that while DUoR remains effective in single-criterion settings, it is less capable of mitigating popularity bias in multi-criteria environments, where the proposed methods provide a more stable and effective solution.

Under the RPI method, as presented in Fig. 10, the performance of the proposed approaches declines compared to the other two methods, with a noticeable drop observed particularly in the aggregate-based scenario (see Fig. 10A). Nevertheless, across all three evaluation types, the proposed RelPref method consistently achieves the highest levels of user satisfaction and system effectiveness, followed closely by PriRelPref. This consistent ranking confirms the superior capability of the proposed methods to capture users’ genuine preferences, even under less favorable popularity determination settings. In the criteria-based analysis (Fig. 10B), Mul, Aug, and Paiw report high novelty and popularity-bias mitigation values but perform poorly in terms of personalized recommendations. DUoR shows moderate performance (Novelty: 0.8922, APLT: 0.8922, LTC: 0.0189). In contrast, the proposed RelPref (Novelty: 0.7666, APLT: 0.7666, LTC: 0.0163) and PriRelPref (Novelty: 0.7736, APLT: 0.7736, LTC: 0.0164) achieve lower debiasing scores while delivering substantially higher user satisfaction. The overall-based analysis (Fig. 10C) closely mirrors these observations. Although baseline methods such as Mul, Paiw, and Aug aggressively mitigate popularity bias and improve diversity, this strategy leads to significant losses in user satisfaction. By contrast, the proposed methods maintain a balanced trade-off among user satisfaction, diversity, and popularity-bias mitigation, and they outperform DUoR in both criteria-based and overall-based evaluations.

For this dataset, the results of the final analysis conducted using the APRI method are summarized as shown in Fig. 11. As shown in Fig. 11A, in terms of user satisfaction, the RelPref method achieves the highest performance, while PriRelPref (Precision: 0.0993, Recall: 0.031, F1-measure: 0.0441) remains at a moderate level. In contrast, the Paiw, Aug, and Mul methods exhibit the lowest performance. Regarding popularity bias mitigation, the observed results are consistent with those obtained from the previous methods and evaluation settings. Among all approaches, PriRelPref demonstrates superior capability in delivering diverse recommendations while maintaining user satisfaction. However, as shown in Fig. 11B, the criteria-based analysis indicates that RelPref achieves the best performance in terms of user satisfaction. Furthermore, RelPref shows strong effectiveness in providing novel and diverse recommendations, as eight out of the top 10 recommended items belong to different categories, and two of these items directly align with the user’s interests and preferences. The overall-based analysis presented in Fig. 11C yields accuracy results consistent with those observed in the criteria-based analysis. However, with respect to the Gini coefficient, the proposed PriRelPref method achieves the same value as in the criteria-based scenario. While the performance of other methods declines in terms of providing diverse items across different categories, PriRelPref maintains stable effectiveness, indicating that its performance remains robust across different analysis types.

In conclusion, the results obtained from the YM20 dataset are notably more satisfactory than those from the previous datasets, reflecting a stronger balance between user satisfaction and the popularity distribution of recommended items. This improvement largely stems from the dataset’s higher density, which provides richer user–item interactions and more informative ratings. As a result, the models capture user preferences more accurately, yielding outcomes that are more stable and interpretable. Furthermore, the higher correlation observed among the multiple criteria indicates that users evaluate aspects such as acting, story, direction, and visuals in a coherent manner. This consistency enhances the reliability of preference learning and produces clearer distinctions across criteria-based, overall-based, and aggregate-based analyses. Overall, the YM20 dataset’s lower sparsity and more consistent user feedback enable the models to achieve a well-balanced performance in terms of accuracy, diversity, and user satisfaction. These findings underscore the importance of data density and evaluation consistency in the effectiveness of multi-criteria recommender systems.

For the final dataset, YM10, the results indicate that, under the $BT{A}_{RPI}$ setting and across all evaluation scenarios, DUoR and RelPref achieve relatively high levels of user satisfaction (Precision $\approx 0.045$–0.053); however, they exhibit lower fairness in terms of item exposure due to elevated Gini values (see Fig. 12). In contrast, Mul, Paiw, and Aug generate extremely high novelty and long-tail coverage, but suffer from substantial accuracy losses, indicating weak alignment with user preferences. Among the proposed methods, RelPref demonstrates a more balanced performance, particularly in the criteria- and overall-based analyses (see Figs. 12B and 12C). RelPref maintains acceptable accuracy while achieving competitive results in diversity and fairness metrics. PriRelPref shows improvements in accuracy in certain scenarios; however, its overall contribution remains limited when compared to DUoR.

Figure 13 presents the analysis results obtained using the $BT{A}_{APRI}$ method. For metrics other than RelPref and PriRelPref, the use of this method results in either stable or declining performance. In contrast, the proposed methods achieve an improvement of approximately 2-3% in overall performance. In the overall-based scenario (see Fig. 13C), user satisfaction reaches its peak in the generated recommendation lists; however, this performance decreases in the criteria- and aggregate-based scenarios, as observed in Figs. 13B and 13A. Nevertheless, despite the observed gains in user satisfaction, a decline in item diversity is also evident.

The comparative evaluation of the aggregate-based, criteria-based, and overall-based analyses using the RPI method on the YM10 dataset shows that RelPref consistently achieves the highest user satisfaction across all evaluation types. As illustrated in Figs. 14A–14C, RelPref outperforms the baseline methods in Precision, Recall, and F1-measure in each analytical perspective. In terms of popularity-bias mitigation, Mul, Paiw, and Aug perform best in the aggregate-based analysis, whereas DUoR, RelPref, and PriRelPref achieve higher Novelty and APLT values in the criteria-based and overall-based analyses, indicating that analysis type strongly influences debiasing outcomes. Diversity fairness also varies across scenarios, with Mul leading in the aggregate-based analysis and PriRelPref achieving the highest Gini values in the criteria- and overall-based evaluations.

As illustrated in Fig. 15, the final evaluation results obtained under the APRI metric indicate that the RelPref methods achieve the highest performance in terms of user satisfaction while adopting a more conservative strategy for mitigating popularity bias. In contrast, Mul, Paiw, and Aug employ aggressive popularity-bias reduction strategies; however, this approach leads to substantial losses in user satisfaction. PriRelPref, on the other hand, demonstrates a balanced performance in both aspects. In terms of Novelty and APLT metrics, beyond the commonly preferred algorithms in the literature, PriRelPref outperforms RelPref and DUoR in delivering new and long-tail items. While Mul ensures the highest diversity in recommendations, PriRelPref closely follows. As shown in Fig. 15B, the criteria-based analysis results resemble those of the aggregate-based analysis in terms of accuracy metrics. Although DUoR achieves relatively higher values in providing new and non-popular items compared to the aggregate-based analysis, this does not translate into success in diversity or user satisfaction. Regarding diversity, the proposed RelPref and PriRelPref methods offer the most diverse recommendations. Finally, as presented in Fig. 15C, the overall-based analysis produces results consistent with both the aggregate- and criteria-based analyses. Paiw, Aug, and Mul display considerably lower performance in user satisfaction metrics. In terms of mitigating popularity bias, the outcomes are similar to those observed in the aggregate-based analysis.

Based on all the evaluation results presented above, the relationship between Precision, which reflects user satisfaction, and APLT, which represents the popularity level of the recommended items, plays a decisive role in evaluating recommendation quality. In this context, the absolute difference between Precision and APLT emerges as a critical indicator of overall system effectiveness. A small difference reflects a desirable balance, where the system maintains high user satisfaction while reducing popularity bias through the inclusion of long-tail items. As summarized in Table 5, the proposed methods—particularly RelPref and PriRelPref—consistently achieve low differences across datasets. In Table 5, bold values indicate the lowest absolute differences between Precision and APLT for each dataset and popularity metric, highlighting methods that achieve a better balance between accuracy and long-tail item exposure. For example, in the YM20 dataset under the RPI strategy, RelPref obtains the lowest value (0.1503), while PriRelPref achieves similarly low differences under the $BT{A}_{RPI}$ (0.1604) and APRI (0.1213) methods. In the YM10 and TA datasets, PriRelPref yields the smallest differences (0.0011 and 0.0191), demonstrating superior balance between user satisfaction and popularity bias mitigation.

These findings indicate that the proposed methods not only prevent the dominance of popular items but also succeed in providing users with more diverse and less popular content. Consequently, RelPref and PriRelPref enhance user satisfaction while also ensuring strong performance in terms of fairness and diversity within multi-criteria recommendation systems. However, it should also be recognized that these improvements in fairness and diversity may inherently come with minor reductions in traditional accuracy metrics. It should be noted that the relatively lower Precision, Recall, and F1 values observed in some cases do not contradict the purpose of a personalized recommender system. Rather, they reflect the well-known trade-off between accuracy and diversity, which is inherent to fairness-aware and popularity-debiasing models. By slightly compromising accuracy, the proposed methods achieve a fairer and more balanced recommendation process that maintains personalization while reducing overexposure to popular items. This balance between user relevance and item diversity is crucial for long-term user satisfaction and sustainable fairness in multi-criteria recommendation systems.

To further clarify this trade-off and provide a consolidated comparative perspective, the accuracy-diversity balance of the proposed and baseline algorithms is systematically summarized in Table 6. The table contrasts the methods in terms of accuracy (Precision, Recall, and F1-measure) and diversity-related metrics (Novelty, APLT, LTC, and Gini Index), offering a holistic view of performance characteristics. For the accuracy dimension, the average of Precision, Recall, and F1-measure is computed for each algorithm; values above the global mean are categorized as High, those below as Low, and values equal or close to the mean as Medium. An analogous procedure is applied to derive the categorical levels of the diversity dimension.

For the YM20 dataset, the results obtained under the three evaluation scenarios show a highly consistent pattern. In this dataset, RelPref and PriRelPref generally achieve high accuracy while maintaining medium levels of diversity across most perspectives. In contrast, baseline methods such as Paiw, Mul, and Aug produce low accuracy but high diversity. This pattern indicates a clear trade-off in baseline algorithms, where gains in diversity come at the expense of substantial accuracy loss. The proposed RelPref and PriRelPref, however, offer a more balanced performance by delivering satisfactory diversity without compromising accuracy. The results for the YM10 dataset exhibit a similar trend. RelPref and PriRelPref once again maintain a high accuracy profile combined with medium-to-low diversity, demonstrating stable and balanced behavior. Baseline algorithms continue to display high diversity but low accuracy, reinforcing the observation that increased data density amplifies accuracy degradation in popularity-biased methods. The TA dataset, being more sparse and noisy, yields lower overall accuracy levels for all algorithms. Nevertheless, RelPref and the DUoR family achieve medium accuracy, preserving competitive performance, while Mul attains high diversity despite a marked reduction in accuracy. PriRelPref also maintains a relatively balanced profile, avoiding extreme deterioration in either accuracy or diversity.

Building on the previous findings, a statistical analysis of APLT and Precision metrics across the TA, YM20, and YM10 datasets further confirms the effectiveness of the proposed RelPref and PriRelPref methods (see Table 7). Across all datasets, the APLT results consistently indicate that the proposed methods significantly improve long-tail item exposure, thereby mitigating popularity bias. In comparisons with Aug, Mul, and Paiw, these improvements are statistically significant ( $p<0.05$) and supported by large to very large effect sizes, while differences with DUoR are generally not statistically significant.

The precision-based analysis yields complementary insights. For all datasets, RelPref and PriRelPref significantly outperform Aug, Mul, and Paiw ( $p<0.001$), with large or very large effect sizes, demonstrating robust gains in user satisfaction. In contrast, comparisons with DUoR do not consistently reach statistical significance, indicating comparable performance between these methods in terms of accuracy. Overall, the statistical results demonstrate that RelPref and PriRelPref achieve a favorable balance by significantly enhancing long-tail exposure without sacrificing Precision, thereby offering a consistent and effective solution for popularity bias mitigation across different datasets.

Building upon these statistical results, it is also essential to examine how user-level dominance influences the behavior of the proposed models. To analyze the effect of dominant criterion consideration on user preference modeling, a comparative evaluation between the base model and the model with dominant criterion is presented in Table 8. The table shows the differences between the two configurations: one that ignores user dominance and another that explicitly incorporates it.

According to the results, the base model achieves higher values for accuracy-oriented metrics such as Precision, Recall, and F1-Measure across all datasets. This indicates that incorporating user dominance slightly reduces the model’s focus on popular and highly rated items, leading to a minor decline in accuracy. However, the model with dominant criterion achieves noticeable improvements in metrics related to novelty, long-tail coverage (APLT and LTC), and Gini index. These improvements demonstrate that considering user-level dominance reduces popularity bias and increases the exposure of less popular but relevant items. Regarding the Gini index, the model with dominant criterion produces lower values in the TA and YM20 datasets, confirming the reduction of popularity bias. In contrast, a slight increase in the Gini Index is observed for the YM10 dataset, which may result from the stronger concentration of dominant user preferences in this data. Overall, the results indicate that incorporating user dominance improves recommendation diversity and fairness, although with a small trade-off in accuracy-based metrics.

In the final stage, the proposed approaches are evaluated through a sensitivity analysis based on the $\alpha$ and $\beta$ parameter values introduced in ‘Relative Preference Model’. Specifically, in order to examine the behavior of the PriRelPref and RelPref algorithms with respect to these two fundamental model parameters, a comprehensive sensitivity analysis is conducted. This analysis aims to assess how variations in $\alpha$ and $\beta$ influence the stability and robustness of the proposed methods under different parameter configurations. The results of this analysis, illustrating the performance variations of both algorithms under different $\alpha$ values $[0,0.5,1,1.5]$ and $\beta$ values $[0.5,1]$, are presented in Fig. 16. While no metric change is observed across all metrics for the S1 and S2 scenarios, only the S3 scenario exhibits variations in certain metrics; these changes are also shown in Fig. 16. In addition, no performance variations are observed for the YM10 and TA datasets under different $\alpha$ and $\beta$ parameter settings. In contrast, sensitivity effects with respect to parameter changes are observed for the YM20 dataset, which represents the densest interaction structure considered in this study. Accordingly, the results of the sensitivity analysis are reported based on the YM20 dataset, where the impact of varying parameter values is more clearly captured.

For the first algorithm, PriRelPref, the analysis indicates that variations in the $\alpha$ parameter do not affect the Precision, F1-measure, Novelty, Gini index and APLT metrics, all of which remain stable across the tested parameter range. In contrast, changes are observed only in the Recall and LTC metrics, as illustrated in Fig. 16B. Among the evaluated metrics, only the LTC metric exhibits noticeable fluctuations, with changes reaching up to 0.11%, after which the metric stabilizes at the parameter configuration $\alpha =1$ and $\beta =0.5$. Additionally, although minor variations are observed in the Recall metric, these fluctuations remain limited to a maximum of 0.16% and eventually converge to a minimal variation of 0.06% at higher parameter values. Similarly, the second algorithm, RelPref, demonstrates parameter-induced variations only in the Recall and LTC metrics, as illustrated in Fig. 16A. The Recall metric shows an average fluctuation of approximately 0.01%, while the LTC metric stabilizes at the same parameter combination ( $\alpha =1$, $\beta =0.5$). When the results of both algorithms are collectively evaluated, it becomes evident that changes in the $\alpha$ and $\beta$ parameters do not produce any meaningful impact on overall performance. This stability suggests that the models generate consistent outputs regardless of parameter adjustments, thereby demonstrating their robustness and reliability for real-world application scenarios.

In addition to its effectiveness in improving diversity and fairness, it is also important to evaluate the computational performance of the proposed approaches. The experimental results summarized in Table 9 indicate that both proposed methods, RelPref and PriRelPref, exhibit high computational efficiency across all datasets and evaluation perspectives. Among the algorithms, those executed on the YM20 dataset record the lowest CPU and clock times, which can be attributed to the smaller data density and the limited number of user–item interactions within this dataset. Conversely, algorithms executed on the YM10 dataset show slightly higher processing times due to its denser structure and larger user and item sets, while the TA dataset yields moderate execution times, reflecting the influence of sparsity on runtime performance.

This study not only evaluated processing times but also the compatibility of the methods with systems with real-time and dynamic item pools. The fundamental operating principle of RelPref and PriRelPref is based on computations that do not require global model training and make decisions solely based on local pairwise comparisons. This feature allows the method to operate without requiring retraining, particularly when a new user joins the system or a new product is added to the catalog. This allows recommendation lists to be updated instantly by performing additional computations only for relevant user-product pairs. Furthermore, the computational complexity of the methods increases linearly with the growth of user-product interactions, supporting scalability and real-time applicability even for high-volume and constantly changing item pools. In these respects, RelPref and PriRelPref are suitable, flexible, and cost-effective methods for updating online recommendation platforms.

Discussion

The empirical analyses conducted across the TA, YM20, and YM10 datasets provide a clear understanding of how the proposed RelPref and PriRelPref algorithms respond to varying levels of data sparsity, rating density, and multi-criteria structures. Across all datasets, both methods consistently achieve a more favorable balance between accuracy, diversity, and long-tail exposure compared to baseline approaches such as Mul, Paiw, and Aug. This balance is particularly meaningful in contexts where traditional models either sacrifice user satisfaction to increase diversity or disproportionately favor popular items, reinforcing popularity bias.

The results from the TA dataset, characterized by high sparsity, show that RelPref maintains accuracy even when user–item interactions are limited. Although PriRelPref performs modestly in aggregated evaluations, it demonstrates substantial improvements in criteria-based and overall-based analyses, suggesting that accounting for dominant user criteria enhances adaptability to heterogeneous rating behaviors. Conversely, methods like Mul and Aug generate highly novel recommendations but fail to align with user interests, underscoring the limitations of overly aggressive debiasing strategies.

In the denser YM20 and YM10 datasets, the benefits of the proposed methods become more pronounced. RelPref consistently yields the highest Precision scores, while PriRelPref excels in fairness and long-tail exposure, as reflected by Gini-based diversity measures. Statistical significance tests further confirm the robustness of these outcomes, with both methods exhibiting large to very large effect sizes across evaluation scenarios. The complementary strengths of RelPref and PriRelPref—one accuracy-oriented, the other diversity-focused—suggest strong potential for hybrid integration in multi-criteria recommender systems.

The superior performance of the proposed methods can be attributed to their (RelPref, PriRelPref) ability to incorporate user-specific contextual information. RelPref models the relative preference ordering of items within a user’s rating profile, capturing fine-grained behavioral signals overlooked by single-criterion or aggregated rating models. PriRelPref extends this approach by emphasizing each user’s dominant criterion, allowing it to better accommodate variations in rating priorities across users. These mechanisms collectively explain the observed improvements in accuracy–diversity trade-offs and long-tail coverage.

Overall, the findings offer meaningful implications for both theory and practice. Theoretically, the study introduces a behaviorally informed framework that integrates relative preference modeling with criterion-aware weighting, advancing existing approaches to mitigating popularity bias in multi-criteria environments. Practically, the computational efficiency and stability of RelPref and PriRelPref demonstrate the suitability of the proposed methods for real-time, large-scale systems. RelPref and PriRelPref maintain fairness while improving personalization performance, which highlights the potential applicability of the proposed framework in domains such as e-commerce, digital streaming services, and decision-support platforms. Together, these insights underscore the practical relevance and theoretical contribution of the proposed methods within the broader recommender systems literature.

Conclusion and future work

This study investigated the impact of popularity bias in multi-criteria recommender systems and evaluated different popularity tendency determination strategies together with multiple recommendation algorithms under diverse multi-criteria evaluation settings. The experimental results consistently show that modeling user preferences in a relative and criterion-aware manner leads to more balanced recommendation outcomes. The proposed methods, RelPref and PriRelPref, demonstrate strong performance by simultaneously improving personalization quality, mitigating the influence of popularity bias, and supporting diversity-oriented recommendation behavior. RelPref is particularly effective in accuracy-oriented scenarios, while PriRelPref achieves more balanced outcomes in terms of fairness and long-tail exposure by incorporating user-specific criterion priorities. These findings indicate that explicitly modeling relative preferences and criterion importance enables recommender systems to better align item strengths with users’ prioritized criteria. Overall, the results confirm that the proposed approaches provide stable and robust performance across different evaluation perspectives and data characteristics, offering an effective mechanism for controlling popularity bias while preserving recommendation quality in multi-criteria settings.

Although the experimental evaluation is conducted on movie-based datasets, the proposed methodology is inherently domain-independent and can be extended to other application areas such as books, music, or e-commerce, provided that multi-criteria ratings are available. Future research will focus on validating the generalizability of RelPref and PriRelPref across diverse user segments and domains with varying popularity distributions. Methodologically, we plan to integrate PriRelPref with neural embedding techniques—including deep contextualized representations and graph-based embeddings—to capture more complex user–item relationships. Another promising direction is the adoption of online and incremental learning strategies, enabling the models to adapt to dynamic environments where user preferences evolve continuously. The use of larger and more heterogeneous datasets will help assess the robustness of the proposed methods under diverse behavioral patterns and sparsity conditions. Additionally, enhancing the explainability and transparency of the methods remains a key priority for strengthening user trust in decision-support environments. Finally, real-world A/B testing with live user interaction will be conducted to validate the practical impact of the proposed models, offering valuable insights into their applicability in operational recommender system settings.

Supplemental Information