A federated learning framework based on transfer learning and knowledge distillation for targeted advertising

Our contribution

Our motivation behind this work is to overcome the challenges posed by isolated data islands and the heterogeneity of data in the advertising industry. We believe that addressing these issues is crucial to advancing our understanding of personalized advertising recommendations, while also prioritizing data privacy, user experience, and advertising conversion rates. To achieve these goals, we propose a federated learning framework called Fed-TLKD, which incorporates transfer learning and knowledge distillation. This approach is exciting because it offers a promising solution to complex advertising problems and opens up new avenues for research and innovation in the field.

• In the context of targeted advertising, this article proposes the integration of federated learning to resolve the issue of isolated data islands. Additionally, the combination of federated learning with transfer learning is presented as a solution to tackle challenges posed by model heterogeneity. Specifically, a transfer learning C-mean clustering algorithm is proposed based on maximum mean discrepancy that better evaluates the distribution differences between local ML parameters and global ML parameters, achieving more effective targeted advertising.

• We propose a dynamic selection algorithm that utilizes knowledge distillation and weight correction to ensure effective participation of various advertisers in the global ML training process. Our innovative approach not only addresses the impact of model heterogeneity, but also guarantees fair participation of each client while optimizing weight assignment to enhance the overall performance of the global model. This algorithm represents a significant advancement in the field of federated learning and has the potential to revolutionize the way we approach collaborative machine learning.

• We put the framework to the test with various datasets and evaluated its performance based on accuracy, loss, and AUC metrics. The results of our experiments speak for themselves, as the Fed-TLKD framework outperforms other models with higher accuracy, lower loss values, and better AUC values, all achieved within the same computation time. These findings demonstrate the potential of our framework to significantly enhance the performance of machine learning systems in real-world applications.

The subsequent chapters of this article are organized as follows: “Literature Review” presents a comprehensive review of related literature in the field, “System Model and Problem Description” provides a systematic introduction to the system model and defines the problem, “Fed-TLKD Framework” delves into the intricacies of the Fed-TLKD framework, explaining its mathematical underpinnings in detail, and “Experimental Result” evaluates the performance of the Fed-TLKD framework using two datasets and compares it with other existing federated learning models.

Machine learning in targeted advertising

In advertising targeting scenarios, machine learning is widely adopted as a method for improving the targeting process (Liu et al., 2021a; D’hooge et al., 2023; Lee et al., 2019; Fang et al., 2022a). In advertising targeting scenarios, machine learning typically centrally receives data uploaded from local models, then distributes the results of the training equally to various advertiser nodes. As Simsek & Karagoz (2019) further enhance the user portrait through using followee/follower property of microblogs, the experimental results indicated a significant level of accuracy. However, Simsek & Karagoz (2019) found that the follower property of microblogs cannot completely represent the user’s preferences because the context differences of social media and social networks also produce different requirements. Therefore, they proposed a novel user model that incorporates various aspects of a user’s message such as text, title, network links, and topic tags, among others, to generate a comprehensive representation of the user (Simsek & Karagoz, 2020). However, Jeong, Kang & Chung (2021) proposed that only considering the user’s preferences on one platform for advertising recommendations is not enough, and it does not meet the expectations of various users. Therefore, the authors proposed an online advertising recommendation system that leverages the user’s consumption history. The system calculates the similarity between users and predicts their preferences for items by comparing the rating history of similar users (Jeong, Kang & Chung, 2021). Zhou et al. (2019) to prevent misleading advertising from stimulating consumption, they set up a three-party game model to analyze the relationship between the government, advertising platforms, and users to prevent users from making any regretful actions due to the launch of misleading advertising.

In summary, machine learning is indeed an effective method in advertising targeting, but in practical application, there is still a certain accuracy gap. Moreover, traditional machine learning approaches have a limitation where central servers have complete control over both the data content and the training process, which raises concerns over the potential risk of privacy breaches.

Federated learning in targeted advertising

The integration of federated learning offers robust technical support for the sharing of data among advertisers, enabling them to make informed decisions and improve their targeting strategies (Zhang et al., 2022a; Zhang & Li, 2022; Vivona et al., 2022). Under the support of federated learning, each advertiser node is very flexible and can independently collect data, and the training process does not require the uploading of real data to the central server. Recently, numerous academic researchers have presented a plethora of federated learning frameworks aimed at enhancing the delivery of targeted advertising. For example, Jiang et al. (2021) proposed a federated sponsored search auction mechanism based on Myerson’s theorem, used for recommending satisfactory ads to users in a short time (Epasto et al., 2021). Epasto et al. (2021) designed a privacy-enhanced solution by combining IBA algorithm and federated learning, which can provide privacy guarantee while achieving competitive performance. However, Liu et al. (2021c) believe that in considering the protection of privacy data, it is also necessary to pay attention to data exchange among advertiser nodes to ensure the revenue of advertiser nodes. Therefore, the authors proposed an optimized construction method for the targeted advertising delivery scenario, based on a federated learning-enabled gradient boosting decision tree (GBDT) model. The method takes into account data islands and irregular and complex flow data, and introduces a new federated voting mechanism called FedVoting. This mechanism aggregates the ensemble of GBDT models that are protected by differential privacy, through multiple training, cross-validation, and voting processes. As a result, it can generate the optimal model that strikes a balance between performance and privacy protection. The experiments indicate that the mechanism has a high accuracy in long-term prediction. However, relying solely on the collected data for user analysis is often not enough, because user data is flowing, changing, and irregular. Therefore, predicting the original advertising click-through rate (CTR) of users is very necessary. Wu et al. (2022) proposed a federated joint original advertising CTR prediction method called FedCTR, which can protect privacy in a way that collects features from cross-platform users.

In the scenario of targeted advertising delivery, the heterogeneous content of data samples and models at different nodes is an important issue. However, most federated learning frameworks overlook this problem, resulting in defects in practical applications. We need to seek more comprehensive and integrated solutions to solve this problem.

Transfer learning in targeted advertising

Ignoring the impact of heterogeneity, the models we train are often difficult to communicate with. At the same time, there are problems such as data sample missing and poor sample quality in the sample dataset, which further damage the quality of the model. To address this issue, many scholars propose that transfer learning can be used (Devi, Namasudra & Kadry, 2020; Zhang et al., 2022b; Fang et al., 2022b; Oslund et al., 2022). For example, Hong et al. (2019), to make advertising recommendations intelligent, understand that videos often come with advertising recommendations, which is an important part of it. Therefore, researchers proposed a model based on transfer learning for intelligent advertising recommendations, and the experimental results indicate that its accuracy is quite impressive (Jiang et al., 2021; Hong et al., 2019). However, due to the complexity and diversity of advertising data, the researchers found that feature transfer alone cannot uncover the relationships between features in advertising data. Hence, Jiang et al. (2021) proposed a click rate method that leverages multi-view feature transfer, and the results of their experiments showed that it outperforms many other advertising click-through rate prediction methods. However, they did not consider the communication time and time cost.

Lian et al. (2019) considered that many personalized advertising recommendation studies had the problem that only some labeled products could be recommended during video playback, which meant that it couldn’t recommend more products that users really liked to them. Therefore, for the image classification model learned on the large dataset, a method is proposed, by transferring the pre-trained deep image classification to solve the scene object recognition problem in the specific task of the model of movies, TV dramas, short videos, and other TV programs, the experiment shows that the model is more effective and efficient (Lian et al., 2019). Manchanda et al. (2019) combine transfer learning and domain adaptation to use the similarity between users to transfer information from users with enough data to users without any active data, thus reducing time costs.

As shown in Table 1, there are many aspects of research on targeted advertising, but a convincing solution for addressing the problems of isolated data island and model heterogeneity caused by the large amount of data in advertising delivery has not yet been found. Additionally, our use of literature analysis tools also confirms this viewpoint. As shown in Fig. 1, in the Web of Science Core Collection, by using “federated learning”, “advertising”, and “transfer learning” to extract all literature from January 2018 to December 2022 and selecting keywords that appeared more than 100 times, we used the VOSviewer tool to generate a relationship graph of these keywords. It is clearly visible that the relationship between “advertising” and the other two keywords is weak, proving that few researches connect the three together.

System model

As shown in Fig. 2, the framework is composed of two entities, the central server, and the advertiser provider. The communication between these two entities only transfers local ML model parameters, while the actual data is only stored locally in the advertiser provider. The functions of these two entities are as follows:

• (1) Central server is the core processing node for personalized advertising delivery and does not receive the original data collected by the advertiser, only receiving local ML model parameters. In our framework, it is responsible for the most difficult personalized federated learning calculations, such as transfer learning, knowledge distillation, and the calculation of the optimal aggregation model, etc. We assume that it has infinite computational power and storage space, in other words, it can support any calculation.

• (2) Advertiser providers are responsible for collecting a large amount of raw data during the customization process of crowd-targeted delivery strategies, and only storing it in local storage units to form a local dataset $D_{local}$. At the same time, we define the public dataset $D_{public}$, the advertiser set A, and a specific advertiser $A_q$, where $q=\left\{1,2,3,\dots \right\}$. The sample data for advertiser $A_q$ is represented as $A_q=X_1^{(q)},X_2^{(q)},\dots ,X_i^{(q)}$, and the label set for these data is represented as $Ta{g}_q=Y_1^{(q)}\dots ,Y_i^{(q)}$. In each round of global iteration, the shared model of traditional FL cannot meet the needs of all advertiser nodes, and problems such as non-independent and identical distribution and heterogeneity exist, making it difficult for advertiser nodes to receive the benefits they deserve in the process of sharing information. Therefore, a personalized learning framework is needed to continuously update and improve FL. The personalized learning framework enables each advertiser to perform personalized training on their local ML model $D_{local}$ to improve personalized accuracy. At the same time, it ensures that different advertiser providers can be allocated the most reasonable returns when cooperating.

Problem description

Assume that there are q advertisers participating in the calculation in the scenario of advertising targeting delivery, and each advertiser has its own local dataset $D_{local}$ at the beginning, which we denote as $\left\{X_i^{(1)}\in D_{local}^1\right\}$, $\left\{X_i^{(2)}\in D_{local}^2\right\}$,…, $\left\{X_i^{(q)}\in D_{local}^q\right\}$, $i=\left\{1,2,\mathrm{3\dots }q\right\}$. At the same time, there also exists a public dataset $D_{public}$ in this scenario. Since advertising targeting delivery data is mostly numerical, we choose long short-term memory (LSTM) as the core model. For the commonly used recursive network model (RNN), there are problems of gradient disappearance and gradient explosion when solving the problem of long-term correlation, and the role of the cell state is to continuously transfer specific information to the RNN. In this way, these problems can be effectively solved, and can be well applied to the scenario of advertising targeting delivery. LSTM is generally composed of forget gates, input gates, and output gates.

The role of the forget gate is to decide to retain or delete the information in the network output $h_{t-1}$ of the previous output in the time series.

(1) $$R_t=\sigma \left(W_r^T\left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_r\right)$$

In this, $W_m$ represents the weight of the forget gate; the superscript T represents transpose; $b_m$ represents the bias of the forget gate; $h_{t-1}$ is the network input at time $t-1$; $x_t$ is the network input at time $t$; $\sigma (\cdot )$ represents the sigmoid activation function, which is used to ensure that the output values are between [0,1], and its expression is:

(2) $$\sigma \left(x\right)=\frac{1}{1+e^{-x}}$$

The role of the input gate is to calculate the weights for preserving and merging information into the state cell, which is to add new sequence information into the state cell. The expression for the input gate is as follows:

(3) $$U_t=\sigma \left(W_u^T\left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_u\right)$$

(4) $$\widetilde{G_t}=tanh\left(W_g^T\left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_g\right)$$

In which, $W_i$ represents the weight of the input gate; $W_c$ represents the weight in the tanh function; the superscript T denotes transpose; $b_i$ represents the bias of the input gate; $b_c$ represents the bias in the tanh function. The expression of tanh(x) is:

(5) $$\tanh \left(x\right)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$$

The expression for the unit state $c_t$ at time $t$ is as follows:

(6) $$g_t=f_t{g}_{t-1}+i_t\widetilde{G_t}$$

The function of the output gate is to determine how much of the information stored in the memory module is passed on to the next time step and the output module. In LSTM model, the output is calculated using the output gate and a tanh function. Therefore, the prediction output at time $t$, $h_t$, is calculated using the output gate and a tanh function.

(7) $$O_t=\sigma \left(W_o^T\left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_o\right)$$

(8) $$h_t=O_t\tanh \left(g_t\right)$$

The function for the prediction accuracy of the personalized learning model can be inferred, where $W_o$ represents the weight of the output gate and $b_o$ represents the bias of the output gate.

(9) $$Accuracy=\frac{1}{N}\sum _{i=1}^N\sigma \left(W_o^T\left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_o\right)/Y_t$$

On this basis, we can calculate the user model loss function of the framework to be:

(10) $$L\left(\omega \right)=L_i\left({\omega }_i,D_{local}^i\right)+\lambda \cdot \sum _{D_{public}}{L}_{KL}\left(h_t^{(i)},Y_i\right)$$where $L_{KL}(\cdot ,\cdot )$ is the Kullback–Leibler divergence. Additionally, by calculating the AUC value for the complete global model used for training, the dataset is set to a total of M positive samples, N negative samples, and M + N predicted ${\hat{Y}}_{t,i}$ to evaluate the model’s performance, $\lambda$ represents the transfer rate. $L_i(\cdot ,\cdot )$ is the local loss.

(11) $$AUC=\frac{{\sum }_{i=1}^{N}1\left\{{\hat{Y}}_{t,i}=Y_{t,i}\right\}}{M}$$

Problem summary

Through the introduction of the above three indicators, we can obtain the unified training goal of our personalized learning framework, through the transfer learning C-means clustering algorithm based on the maximum mean difference to accurately classify advertisers and the dynamic selection algorithm to eliminate asynchrony, so we can get the maximum accuracy and the minimum loss value, which is also expressed by the follows.

(12) $$max\left\{Accuracy=\frac{1}{N}\sum _{i=1}^N\sigma \left(W_o^T\left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_o\right)/Y_t\right\}$$

(13) $$min\left\{L\left(\omega \right)=L_i\left({\omega }_i,D_{local}^i\right)+\lambda \cdot \sum _{D_{public}}{L}_{KL}\left(h_t^{(i)},Y_i\right)\right\}$$

(14) $$AUC=lim\frac{{\sum }_{i=1}^{N}1\left\{{\hat{Y}}_{t,i}\right\}}{M}\to 1$$

M is the number of positive samples, which means serving the right amount of ads.

Faced with the heterogeneity of various aspects among advertisers and the implementation of targeted advertising delivery, the proposed Fed-TLKD is expected to achieve higher accuracy, lower loss values, and better AUC values. This will lead to a faster convergence between local models and global models and significantly improve the efficiency of personalized learning models.

The symbols used in this study are shown in Table 2.

Overall introduction of the framework

The Fed-TLKD framework is divided into two modules: Advertiser-C-means clustering with transfer learning based on maximum mean difference, and Central Server-federated learning algorithm based on knowledge distillation and weight correction.

• Our approach involves the utilization of transfer learning and clustering optimization algorithms, which are executed by an access point (AP). This approach is particularly relevant to ad targeting datasets, which may contain text data describing the ad content or evaluations of targeted users. Text data in this field have unique characteristics, and we have developed a feature-based transfer learning training algorithm to ensure that global ML parameters in different fields do not conflict and that scarce private data does not negatively impact experiments. Our approach emphasizes privacy protection while still achieving excellent results through optimized algorithms.

• Our solution to the challenges of federated learning involves the implementation of a personalized federated learning algorithm. Traditionally, the effectiveness of federated learning relies heavily on the similarity of data distributions among clients and identical model architectures. However, in real-world scenarios, client data is often non-independent and non-identically distributed, which can result in global ML models trained through federated learning (FL) failing to meet the individual needs of each client or guarantee maximum benefit for all participating clients.

To overcome this challenge, our personalized federated learning algorithm leverages a two-pronged approach. Firstly, we compute an optimal aggregate model that takes into account the unique data distribution of each client. This ensures that the global model trained through FL is customized to the specific needs of each participant. Secondly, we update personalized model parameters based on the unique data of each client. This helps to further optimize the model for each individual participant and ensure maximum benefits are achieved.

Advertiser provider-C-means clustering with transfer learning based on maximum mean difference

Central server-federated learning algorithm based on knowledge distillation and weight correction

Dataset description

We use two advertisement click-through rate prediction datasets, Ali-Display-Ad-Click (ADAC) and click-through rate data (CTRD), which are respectively sourced from the open data platform Alibaba Tianchi and the Kaggle online advertisement click-through rate prediction competition. The ADAC dataset includes 20 fields and 704,319 pieces of data, with 80% of the dataset being used as the training dataset and the remaining 20% being used as the test dataset. Gai et al. (2017) used this dataset to propose an industrial strength solution with a model named Large Scale Piece-wise Linear Model (LS-PLM) for the main CTR prediction in an advertisement system. Zhou et al. (2018) proposed a deep interest network (DIN) for online advertisement and other industrial applications for advertisement click-through rate prediction. The CTRD dataset includes 17 fields and a total of 80,000 training data and 20,000 test data. These two datasets both predict the click-through rate of online advertisements and can well reflect the effectiveness of the advertisement being delivered to the designated audience, thereby verifying the ability of the Fed-TLKD model. Therefore, we choose to use these two datasets as the experimental datasets.

Experimental environment and experimental parameter settings

Experimental results

The acc, loss, AUC results and computational time cost

Accuracy results

To better evaluate the performance of our framework, we compared the accuracy of the Fed-avg, Fed-sgd, Fed-prox, Fed-TLKD without CLTMMD (In response to: “Experimental Environment and Experimental Parameter Settings” C-means clustering with transfer learning based on maximum mean difference), Fed-TLKD without KDWC (in response to: “Experimental Results” federated learning algorithm based on knowledge distillation and weight correction), and Fed-TLKD. As shown in Fig. 3, the performance of the Fed-avg (Chen, Orekondy & Fritz, 2020), Fed-sgd (Mudd et al., 2022), and Fed-prox (Kim & Hospedales 2023) models remains poor with an increasing number of communication iterations of the global ML model. The Fed-TLKD without CLTMMD and Fed-TLKD without KDWC have good accuracy compared to the other three federated models, there is a lot significant improvement in the results. Only Fed-TLKD has constantly rising accuracy, and its precision is relatively high. Compared to other frameworks, the highest difference in model accuracy can reach 73%. The formula for maximum ACC is as follows:

(35) $$max\left\{Accuracy=\frac{1}{N}\sum _{i=1}^N\sigma \left(W_o^T\left[\begin{array}{c}{h}_{t-1}\\ x_t\end{array}\right]+b_o\right)/Y_t\right\}$$

Loss Results

To better evaluate the performance of our models, we compared the loss values of Fed-avg, Fed-sgd, Fed-prox, Fed-TLKD without CLTMMD, Fed-TLKD without KDWC, and Fed-TLKD. As shown in Fig. 4, the performance of Fed-avg and Fed-sgd did not significantly improve with the increase in the number of communication iterations of the global ML model and they performed poorly. Fed-prox improved further, but there was still a significant difference. The Fed-TLKD without CLTMMD and Fed-TLKD without KDWC showed a consistent declining trend, but their performance still lagged behind that of the Fed-TLKD. The formula for minimum loss is as follows:

(36) $$min\left\{L\left(\omega \right)=L_i\left({\omega }_i,D_{local}^i\right)+\lambda \cdot \sum _{D_{public}}{L}_{KL}\left(h_t^{(i)},Y_i\right)\right\}$$

AUC results

AUC accurately reflects the relationship between the true positive rate and false positive rate of a certain learner and represents the overall accuracy of detection. We also compared the above six frameworks. As shown in Fig. 5, it is clearly seen that the area under the curve of the Fed-TLKD is larger than the other five frameworks. The formula for AUC is as follows:

(37) $$AUC=lim\frac{{\sum }_{i=1}^{N}1\left\{{\hat{Y}}_{t,i}\right\}}{M\times N}\to 1$$

Computational time cost

The huge amount of data also affects the quality of frameworks, so being able to reasonably control the time cost also becomes an important index.

As shown in the results of Table 4, the communication time cost of the Fed-TLKD is almost the same compared to the other five, and the communication time cost of the remaining five models does not have much difference. Therefore, it can be proved that our Fed-TLKD framework is indeed better than the other five models.

Table 4:

Time cost.

Training time for local ML models (unit: seconds)	ADAC	CTRD
Fed-avg	4.9876	7.6481
Fed-sgd	4.9628	7.4572
Fed-prox	4.5263	7.2458
Fed-TLKD without CLTMMD	3.2658	6.1584
Fed-TLKD without KDWC	3.2494	6.2673
Fed-TLKD	3.2821	6.4862

DOI: 10.7717/peerj-cs.1496/table-4