Generating, retrieving persona and generating responses for long-term open-domain dialogue

Response generation in long-term dialogue

The purpose of open-domain dialogue systems is to facilitate natural and engaging conversations between humans and machines across a wide range of topics and contexts. Recently, the advancement of open-domain dialogue systems has been propelled by availability of large-scale dialogue datasets and the development of transformer-based language models, such as Meena (Adiwardana et al., 2020), DialogueGPT (Zhang et al., 2020), BlenderBOT (Roller et al., 2021), and PLATO (Bao et al., 2020), trained on large-scale corpora. Despite these advancements, recent models are constrained by the maximum input length, which is typically 1,024 tokens, preventing them from fully utilizing historical contexts in multi-session conversations such as MSC. To address this limitation, recent approaches have employed techniques such as summarizing previous dialogues or retrieving past contexts.

In the case of summarization models, manually summarized data are required, making it challenging to choose which elements should be included in the summary. Long dialogue summarization datasets are more difficult to collect as they consist of thousands of input words, which still necessitate long input sequences (Zhong et al., 2021; Chen et al., 2022). Recently, a method for automatic summarization with large language models (LLMs) has also been proposed (Wang et al., 2023). However, human evaluation is still required for verification of hallucinations and consistency (Vakharia et al., 2023).

Another case of retrieval-based models involves retrieving sentences relevant to the current utterance within the dialogue history to generate responses (Ramakrishnan et al., 2022; Han et al., 2021; Shu et al., 2023). These models rely on extracting specific pieces of information from the dialogue history that are directly related to the current context, aiming to improve the relevance and coherence of the generated responses. However, this approach is inherently limited by its dependence on explicit information present in the dialogue history. If the required context or details are implicit or scattered across multiple turns of the conversation, the retrieval model may struggle to accurately identify and aggregate the necessary information. Additionally, navigating extensive dialogue histories increases computational complexity and poses challenges in efficiently retrieving the most pertinent information. For instance, Chen et al. (2024) highlight that retrieval-based methods face difficulties in managing growing memory databases and ensuring the retrieval of relevant memories, especially as conversations accumulate over time.

Use of persona in dialogue generation

In open-domain dialogue research, generative models are trained on a vast amount of diverse utterances from many individuals. Consequently, the model encounters the issue of not generating consistent sentences (Li et al., 2016). To address this issue, dialogue datasets containing pre-defined personas, such as PersonaChat (Zhang et al., 2018) and Multi-Session Chat (Xu, Szlam & Weston, 2022), were proposed. However, using pre-defined personas cannot be dynamically updated during the chat. Moreover, creating such datasets requires significant resources. Therefore, recent research has focused on methods enabling models to dynamically extract or infer personas from dialogue history. Wu et al. (2020) introduced a two-stage attribute extractor, which utilizes the Persona Chat dataset and the Dialogue Natural Language Inference (DialogueNLI) dataset (Welleck et al., 2019) to extract persona attribute triplets from dialogues. However, Zhu et al. (2023) identified inconsistencies and ambiguities in the annotations of the DialogueNLI dataset. To address this, they proposed PersonaEXT, a dataset with better-annotated data. Nevertheless, PersonaEXT lacked a robust framework for handling implicit persona attributes and did not fully leverage contextual information from ongoing utterances during persona extraction. PLATO-LTM (Xu et al., 2022) extracted persona sentences from each utterance and utilized them to generate responses. However, this method also limits the extraction of persona that is not explicitly manifested in utterances. To obtain the persona not explicitly manifested, Wang et al. (2022) used GPT2 to infer the persona. This method also restricts the ability to infer personas with diverse expressions due to its reliance on a fixed set of entities and relation types. To overcome these limitations mentioned above, we use the ability of Llama 2 in-context learning to generate the persona of each speaker in real-time with diverse expressions.

Persona retrieval in dialogue generation

In persona dialogue research, two methods are commonly used for incorporating persona into response generation. The first method involves inputting all persona sentences into the generation model and handling them internally within the model. One of them applies an adaptive attention mechanism within the generation model to integrate information between persona and context (Huang et al., 2023). The second method employs a retrieval model to fetch relevant persona sentences. Modular Prompted Chatbot (Lee et al., 2023) employs the DPR model to find persona-like information relevant to the current utterance. However, the DPR model used in the previous study (Lee et al., 2023) was trained on pairs of Wikipedia passages and utterances (Feng et al., 2021), making it unsuitable for retrieving relevant persona sentences. One study (Xu et al., 2022) proposed the DuLeMon dataset, which annotates the persona relevant to the response. They trained their retrieval model using the DuLeMon dataset to retrieve relevant persona sentences. However, the DuLeMon dataset is in Chinese. Annotating it also takes a considerable amount of time. In this article, we propose a new dataset called PUP, created with minimal use of human annotation. Additionally, we train the model using this dataset to retrieve persona sentences necessary for response generation.

Multi-session chat

A conversation in the multi-session chat (MSC) (Xu, Szlam & Weston, 2022) consists of the current session, representing the ongoing conversation, and several previous sessions. We denote the MSC dataset D as a list of N conversations in the format of (H, C, R). Here, H = H¹,H²,⋯,H^M and H^M denotes the conversation of the M-th session. Hⁱ = $h_1^i$, $h_2^i$,⋯,$h_l^i$, where each element represents individual history utterances. C = c₁,c₂,⋯c_n. It represents n utterances of the current conversation session, where c_n denotes the current utterance. Our task is to generate persona from each element in all Hⁱ, retrieve relevant persona sentences from the generated persona list, concatenate them with the current session, and generate response R.

Persona generation

To summarize long-term dialogues from previous sessions into persona sentences, we generate the persona of each speaker in the dialogue using Llama 2 in-context learning capability. Although the GPT-3.5 model may generate persona effectively, we decided to use the freely available Llama 2 model considering the subscription cost for the GPT-3.5 model. First, we define the persona to be generated from the dialogue using the format “subject is related with object”. In this structure, “is related with” serves as a placeholder for verbs that describe the nature of the interaction or connection, such as “has”, “likes”, “doesn’t have”, or “went”. This flexibility allows the format to encompass various types of relationships or actions expressed in the dialogue. The purpose of using this standardized format is to simplify the data refinement and management process during post-processing by reducing persona sentences into a unified structure. This format is also included in the prompt to guide the persona generation process. Next, we direct the Llama 2 model to generate persona for the previous session, using the persona directly generated by humans from the sample dialogue as a one-shot example. The reason for including only one example is that more examples could lead to exceeding the maximum sequence length of Llama 2, which is 1,024 in our settings. Finally, after generating the persona from every previous session dialogue with Llama 2, we proceed with post-processing. Since Llama 2 is a generative model, unnecessary sentences or words such as “thanks” might be generated along with the persona. These unnecessary sentences or words need to be removed for the next step, which is the persona retrieval model; we utilize regular expressions to remove them from the generated sentences. All processes described above, including actual prompts used, can be seen in Fig. 3.

Persona retrieval

Instead of using all generated persona sentences from the list, we retrieve a few persona sentences that are most relevant to the current utterance. Initially, we approached this task as a multiple-choice problem where single or multiple correct responses (Liu et al., 2018) were selected from a list of the generated persona. We provided utterances and generated persona sentences to Llama 2, prompting it to select relevant options. However, in all k-shot settings (0 ≤ k ≤ 3), we notice that the chosen response by Llama 2 changes when choices are rearranged, and it frequently provides incorrect answers each time.

PUP (Persona-Utterance Pair Dataset). We construct a new PUP dataset, which consists of pairs of utterances and personas related to the utterance. We then fine-tune the Bi-Encoder model on the PUP dataset to retrieve relevant persona sentences. To construct the PUP, we first group consecutive utterances in the conversation into pairs as (c_i,c_i+1). Next, using Llama 2, we generate persona p_i+1 from utterance c_i+1 to construct pairs (c_i,p_i+1). Finally, we apply regular expressions during post-processing to remove sentences or words that are irrelevant to the generated persona. Subjects “speaker 1” and “speaker 2” are also removed to address the issue of high similarity caused by these subjects during subsequent training or inference processes. The aforementioned processes are applied to the MSC session 1, resulting in the generation of over 47,000 positive pairs. Of these positive pairs, 80% are used as the training set and, 20% are used as the validation set.

Training procedure. The dense passage retrieval (DPR) model (Karpukhin et al., 2020) proposed by Facebook adopts a bi-encoder architecture. It compares similarity between a question and a Wikipedia passage to search for passages relevant to the question. Inspired by DPR, we use Current Utterance Encoder E_c(⋅) to encode the utterance c_i, and use Persona Encoder E_p(⋅) to encode the persona p_i+1.

Encoder E_c and E_p are initialized with the standard BERT pre-trained model (Devlin et al., 2019) and then fine-tuned on our PUP dataset. For each training sample, we define the positive pair $p_{i+1}^{+}$ as generated persona from the next utterance and negative pair $p_{i+1}^{-}$ as sampled randomly from persona generated of different dialogues. Given the utterance c_i, positive persona $p_{i+1}^{+}$, and negative persona $p_{i+1}^{-}$ are used to tune our network. We optimize the loss function as the negative log-likelihood of the positive persona: ${\displaystyle L\left(c_i,p_{i+1}^{+},p_{i+1,1}^{-},\dots ,p_{i+1,n}^{-}\right)=-log\left(\frac{e^{\text{sim}\left(c_i,p_{i+1}^{+}\right)}}{e^{\text{sim}\left(c_i,p_{i+1}^{+}\right)}+\sum _{j=1}^n{e}^{\text{sim}\left(c_i,p_{i+1,j}^{-}\right)}}\right).}$

We define similarity as the dot product of each encoder’s output: ${\displaystyle \text{sim}\left(c,p\right)=E_c{\left(c\right)}^T{E}_p\left(p\right).}$ Inference procedure. Once the persona generation process is complete, a persona list L_p = ρ₁, ρ₂, ⋯, ρ_n containing n persona sentences is generated from the history session. Since the persona list contains persona sentences for both Speaker 1 and Speaker 2, we separate them into two lists: L_sp1 and, L_sp2. During the inference time, when the speaker of the current utterance is Speaker 1, we select the persona list L_sp2, and vice versa. Therefore, when the speaker of the current utterance is Speaker 1, the Current Utterance Encoder encodes the current utterance, while the Persona Encoder dynamically encodes each element of the persona list L_sp2. In the end, we utilize a threshold value p_thres to select all persona sentences for which the dot product result between E_c(c) and E_p(p) exceeds the threshold. These selected persona sentences are then added to the list L_rt. After conducting a few experiments to compare different threshold values, we found that setting p_thres = 0.68 achieved the best performance. This value showed the highest similarity to the ground truth based on ROUGE-N and BLEU-N scores. Lower threshold values tended to include irrelevant personas, while higher values missed potentially useful personas.

Response generation

During training, when a relevant persona list L_rt is retrieved along with the current utterance, the model generates a response based on persona sentences and current session dialogue. To separate each persona sentence, we use ‘|’ as a separator and insert the special token <sep > between persona sentences and the current utterance. Within the utterance, we use special tokens <sp1 > and <sp2 > to distinguish between speakers. With the input format mentioned above, encoder encodes them into a hidden vector h_n: ${\displaystyle h_n=Enc\left(Emb\left(L_{rt};c_1;c_2;\cdots ;c_n\right)\right).}$ The response decoder, taking h_n as input, generates the response $\widehat{R_t}$ token by token: ${\displaystyle P\left({\hat{r}}_{t,k}\right)=Dec\left({\hat{r}}_{t,k}|{\hat{R}}_{t<k},h_t\right).}$

The cross entropy loss between the generated response $\widehat{R_t}$ and the ground truth response R_t is calculated as: ${\displaystyle \mathcal{L}=\sum _{R_t\in D}CELoss\left(\widehat{R_t},R_t\right).}$

Implementation details

Persona generation. To generate persona, the llama-2-13b-chat model is used without any fine-tuning.

Persona retrieval. The bi-encoder, consisting of the persona encoder and the utterance encoder, is initialized from a standard BERT pre-trained model with 12 attention heads and 768 hidden sizes. For the PUP datasets, we train both encoders for up to 40 epochs using Adam (Kingma, 2014) optimizer with a learning rate of 10⁻⁵, a batch size of 16, and a dropout ratio of 0.1. The fine-tuned model is trained with a maximum of three 24GB GPUs (NVIDIA GeForce RTX 3090).

Response generation. All baseline models are initialized from a publicly available model. For the MSC and the Conversation Chronicles datasets, we train all baseline models for up to 40 epochs using AdamW (Loshchilov & Hutter, 2018) optimizer with a learning rate of 10⁻⁵ and, a batch size of 16. The weight decay for the AdamW optimizer is set to 0.01. GPUs with identical specifications as described above are used.

Baseline methods

To demonstrate the hypothesis that “summarizing previous dialogues into each speaker’s persona while retaining recent dialogues aids response generation”, we use the method of concatenating as many dialogues as possible within the model’s maximum input length as the baseline and compare it with our proposed GRGPerDialogue Framework. The baseline generation model is described as follows.

• GPT2: GPT2 is a transformer-based language model developed by OpenAI, capable of generating human-like text. There are four sizes: GPT2-SMALL, GPT2-MEDIUM, GPT2-LARGE, and GPT2-XL. Their parameters are 124M, 355M, 774M, and 1.5B, respectively. We choose GPT2-MEDIUM, and GPT2-LARGE as baselines.

• BART: BART is a transformer-based encoder–decoder model introduced by Facebook AI. It is designed for various natural language processing tasks such as text generation and summarization. There are two different sizes: BART-base and BART-large. Their parameters are 140M and 400M, respectively. We choose BART-base in this study.

Automatic evaluations

Automatic evaluations used in our analysis are BLEU-n (Papineni et al., 2002) and ROUGE-n (Lin, 2004). BLEU score serves as a metric for assessing the quality of machine-generated text by comparing it against one or more references. It quantifies the precision of overlapping n-grams between generated text and references. The BLEU-1 metric we use evaluates word-level matching, while BLEU-2 evaluates the matching of consecutive two-word sequences. Similarly, ROUGE is one of the metrics used to evaluate the quality of generated text by comparing it with reference text. A notable distinction from BLEU is that ROUGE measures the count of overlapping n-grams between the generated text and the reference. The ROUGE-L metric employed in our analysis is predicated on the length of the Longest Common Subsequence (LCS) between the generated text and the reference.

Human evaluations

Evaluating the quality of open-domain dialogue is considered challenging. Reference-based automated evaluation metrics, such as BLEU and ROUGE, can assign low scores even to appropriate responses because they simply compare how much they overlap with actual reference answers (Liu et al., 2016). Therefore, human evaluation is desirable to thoroughly verify the quality of conversation across various aspects. To conduct human evaluations, we randomly sample 100 dialogues from the MSC test set. We provide all the sessions and responses generated by each model to four proficient English-speaking annotators. We ensure that these annotators do not know which model generated which response. We then instruct them to assign scores from 0-4 to the responses based on the following criteria:

• Fluency for measuring the smoothness and naturalness of generated responses. Annotators primarily evaluate how smoothly and naturally the model continues the dialogue, checking for grammatical errors or awkward expressions.

• History relevance for measuring how well the responses align with the dialogue history. Annotators evaluate whether the model appropriately reflects the context of previous dialogues.

• Consistency for measuring the extent to which a model’s responses maintain consistency in topics, characters, settings, etc. Annotators evaluate whether the model generates consistent responses in the same topic or situation and whether there are any contradictions or inconsistencies between responses.

Automatic evaluation

First, we compare baseline models trained by concatenating the dialogue as much as possible with those trained by summarizing previous sessions’ dialogue into persona sentences and concatenating only the current session’s dialogue. We also compare the use of a pre-defined persona and a generated persona for this summarization. In this experiment, we use the same persona retrieval model trained on the PUP dataset to compare only the effect of persona generation.

As the dialogue gets longer, summarizing previous sessions’ dialogue into each speaker’s persona helps with generating responses. As shown in Table 2, the average token count of dialogues (e.g., Dialogue’s Ave. Tokens: 444.84 in Session 2 and 1,598.78 in Session 5) is significantly higher than the token count of pre-defined persona sentences (e.g., Pre-defined Persona Ave. Tokens: 141.39 in Session 2 and 479.49 in Session 5). This indicates that pre-defined persona sentences effectively reduce the dialogue length to approximately one-third, preserving essential information. As demonstrated in Table 3, despite reducing the dialogue length, pre-defined persona sentences still yield comparable or even better performance in longer sessions. For instance, in Session 4, using pre-defined persona sentences with GPT2-medium achieves a BLEU-1 score of 16.13, outperforming the baseline score of 15.35. These results highlight that pre-defined persona sentences are particularly beneficial in mitigating the truncation-induced information loss that occurs in longer dialogues. In our setting (with a model’s maximum sequence length of 1,024), most dialogues begin truncating earlier utterances around Session 4. This truncation leads to inevitable information loss in baseline models, whereas pre-defined persona sentences help retain the core context and improve response generation. Consequently, pre-defined persona sentences demonstrate their effectiveness not only in reducing token counts but also in enhancing model performance as dialogues become longer.

The generated persona is much more powerful than the pre-defined persona. We find that using the Llama 2 inference ability, we can generate intrinsic persona not explicitly present in the dialogue, which significantly aids in response generation. As shown in Table 2, the average token count for generated persona sentences (e.g., 156.07 in Session 2 and 529.35 in Session 5) is consistently higher than that of pre-defined persona sentences (e.g., 141.39 in Session 2 and 479.49 in Session 5). Despite this increase in token count, as can be seen in Table 4, using generated persona resulted in better performance across all models. For instance, when applying generated persona to the BART model, an average improvement of 1% in BLEU-1 score is observed across all sessions compared to baseline models. In our opinion, this is because the generated persona contains richer historical information, such as intrinsic persona. Table 5 shows a case study of the pre-defined persona and generated persona used in the training. The generated persona sentence “Speaker 1 is currently experiencing hot weather but likes winter” contains more information than the pre-defined persona sentence “I like winter”. Additionally, the generated persona sentence above is an intrinsic persona that cannot be obtained solely from Speaker 1’s utterance “Yeah, I like winter, though I do not go outside”, but rather requires inference from the preceding utterance by speaker 2, “hot enough for you? can not wait for winter”.

The GRGperDialogue framework is effective even in a conversation where pre-defined personas are not provided. We also conduct experiments with the Conversation Chronicles dataset, which does not provide a persona. While the experimental setup mirrors that of the MSC dataset, experiments incorporating pre-defined persona are omitted. As shown in Table 4, our proposed framework achieves better results than all baseline models. However, we also observe that the extent of performance improvement is generally lower than when applying the MSC dataset. We believe that this is because, while constructing the MSC dataset, two crowd workers generated conversations based on a pre-defined persona, resulting in conversations containing many persona elements.

Additionally, experiments are conducted to assess effect of the persona retrieval on response generation. In these experiments, we compare responses generated by concatenating all personas with those generated only with the retrieved persona.

The process of retrieving a persona relevant to the utterance helps generate responses. As seen in Table 6, the retrieval process shows better results in all cases. This means that it is more advantageous to utilize the persona retrieval model rather than relying solely on the inference ability of the generation model when generating responses. In particular, we find that performance decreases without using the retrieval process as the session increases. We believe this is because, as the number of sessions increases (an average of five persona sentences of each speaker per session), the amount of generated persona information also increases, potentially leading the model to become confused about which persona sentences to use when generating responses.

The retrieval model trained on the PUP dataset effectively retrieves persona sentences relevant to the current utterance. A study on the Modular Prompted Chatbot (Lee et al., 2023) utilized a DPR model, named MultiDoc2Dial (Feng et al., 2021), to retrieve historical dialogue information relevant to the current utterance. However, the MultiDoc2Dial model was trained on pairs of Wikipedia passages and utterances, making it unsuitable for retrieving relevant persona sentences. To evaluate our proposed retrieval model, we compared the results of the MultiDoc2Dial model with our retrieval model. Initially, we examine persona sentences retrieved by each retrieval and their similarity scores. Our findings indicate that both models exhibit a degree of proficiency in persona retrieval. However, as seen in Table 7, the retrieval model proposed in a previous study (Feng et al., 2021) assigns high scores even to unrelated persona, posing a challenge in setting a threshold because of the narrow score margin between related persona and unrelated persona. Unlike MultiDoc2Dial, where the similarity scores are narrowly distributed and make thresholding difficult, our model produces more distinguishable similarity scores between relevant and irrelevant persona candidates. Based on qualitative analysis, we observed that scores below 0.6 often corresponded to weakly related or irrelevant persona sentences, while scores above 0.7 sometimes excluded valid persona sentences. We therefore focused on the 0.6–0.7 range and evaluated thresholds in increments of 0.02. Based on BLEU scores, we selected 0.68 as the optimal threshold. To illustrate the impact of the threshold on generation performance, we report BLEU-1 scores from Session 4, where the score differences across thresholds were most prominent. As shown in Table 8, the performance peaks at 0.68, supporting our choice of threshold.

Human evaluation

In addition to automatic evaluation, we conduct human evaluation on the responses generated by BART-base. The reason for this is that BART achieves the highest performance in automated evaluation of generated responses. Results of the human evaluation are summarized in Table 9. Generally, our proposed method outperforms all baseline methods across all perspectives. In terms of fluency, all models demonstrate similar performance, which means BART models can generate fluent and grammatically correct responses. In terms of history relevance, our method achieves the best performance. The overall scores, however, are relatively low, primarily because only a small portion of responses in the dataset require historical context. Many responses—such as greetings or topic-initiating utterances—do not rely on prior context, leading annotators to assign lower scores in those cases. Similarly, our method achieves the best result in consistency, indicating that generated responses are non-contradictory.

Qualitative example

Table 10 presents a qualitative case study demonstrating how generated and retrieved personas influence BART’s response generation. The example involves a dialogue where the speaker mentions majoring in computer engineering, having a passion for programming, and aspiring to start a company with a friend.

This case is analyzed from the perspectives of fluency, history relevance, and consistency. In terms of fluency, all model responses are grammatically correct and natural, showing no significant difference across conditions. However, notable differences emerge in history relevance. Without persona information, the model generates a generic response such as “just a normal day at school,” which lacks contextual relevance to the current question, “How is school going?” When using only the generated persona, the model includes unrelated information such as “starting a company,” which does not directly answer the question. In contrast, combining generated and retrieved personas results in a response that accurately reflects the context, referencing “time on the computer” and “engineering assignments,” which are more aligned with the speaker’s current situation. A similar trend is observed in consistency. The use of only generated personas may introduce off-topic content that weakens coherence, while incorporating retrieved personas helps the model stay more aligned with the speaker’s established identity and earlier utterances. This example illustrates the limitations of relying solely on generated persona—which can introduce irrelevant details—and the effectiveness of combining it with retrieved persona to improve both the history relevance and consistency of generated responses.