Generative named entity recognition framework for Chinese legal domain

Sequence tagging method

Sequence tagging method is the classic method to tackle the NER task, which views the NER task as a token-level classification task, assigns a tag for each token from a pre-designed tagging scheme and utilizes conditional random field (CRF) or tag sequence generation methods to decode and obtain the NER results. Xiaofeng, Wei & Aiping (2020) present an incorporating token-level dictionary feature method, making it easier to quantify the effect of dictionary features and decoupling dictionary features from the external dictionary during the training stage. Li et al. (2021) combines the ordered neurons long short-term memory (LSTM) network (Shen et al., 2018) with the BERT model and CRF layer to enhance model’s ability of mining the hierarchy information of text, which can improve the effect of model in NER task in the legal domain. Shi et al. (2022) combines the bidirectional LSTM (BiLSTM) network with RoBERTa model and CRF layer to tackle Chinese NER task in legal domain, which achieves significant effect on legal NER. An et al. (2022) have designed an improved character-level representation approach to enhance the specificity and diversity of feature representations, and propose a multi-head self-attention based BiLSTM-CRF (MUSA-BiLSTM-CRF) model to tackle the Chinese clinical NER task, which achieves the best performance on two CCKS challenge benchmark datasets. Deng, Lin & Lu (2023) propose a multi-feature fusion model for Chinese named entity recognition that enhances BERT by fully utilizing information across radicals, words, and lexical levels, thereby addressing the inefficiencies in Chinese NER concerning the use of radicals and words, and achieve better F1 values on the Weibo dataset and OntoNotes4.0 dataset. Wang et al. (2024) propose a lexicon-enhanced method for recognizing Chinese long named entities that integrates a SkipWord-Lattice module, which resolves issues with word overlap, redundancy, and confusion, with a Word-Aware Attention module that enhances the interaction between character tokens and word tokens, and achieve state-of-the-art (SOTA) performance on four Chinese NER datasets.

Span-based method

Span-based method views the NER task as a span-level classification task, which aims to find all possible entity spans in the text and assign entity type for each span. Compared to sequence labeling methods, span-based methods are better able to extract nested entities. Tan et al. (2020) propose a boundary enhanced neural span classification model, which views the NER task as boundary detection task and span classification task, and achieves the SOTA performance in terms of F1 on the ACE2004, ACE2005, and GENIA datasets. Yu, Bohnet & Poesio (2020) utilize the bi-affine mechanism to calculate span classification scores in NER task, which can enhance the ability of model for locating entity spans, and achieve SOTA performance on eight corpora. Huang et al. (2022) design a span selection framework with generative adversarial training, which demonstrates the excellent performance on four nested NER datasets. Zhang & Chen (2023) propose a simple yet effective span-based model with knowledge distillation to preserve memories and multi-Label prediction to prevent conficts in continual learning for NER. Zheng et al. (2024) propose a lexicon-free Chinese NER framework called SENCR that incorporates a boundary detector for boundary supervision, a span-convolutional network for better span representation and classifcation and a novel counterfactual rethinking strategy in inference for debiased boundary detection, and prove the effectiveness of SENCR on four Chinese NER datasets.

Recently, some studies have designed new architectures or incorporated different paradigms to tackle the NER task. Li et al. (2020) formulate the NER task as a machine reading comprehension task, and extract possible entity spans by answering the queries given the contexts. Yan et al. (2021) model the NER task as a sequence generation task, and obtain the NER results in a generative way. Li et al. (2022a) propose a joint training multi-task model for multi-answer types of reading comprehension task by adopting a feature-based paragraph extraction mechanism to extract more relevant sentences in the passage. Li et al. (2022b) predict the relation between word and word in the sentence by modelling the NER task as word-word relation classification, and then extract entities by decoding based on the relation between word and word. Lu et al. (2023) recast the NER task to a seq2seq task where a prefix language modelling objective is introduced to reduce the gap between pre-training and fine-tuning. Shen et al. (2023) propose a generative approach for NER that converts the task into a boundary denoising diffusion process, which achieves comparable or better performance on six nested and flat NER datasets. Zhang et al. (2023a) propose a novel Decomposing Logits Distillation (DLD) method, which is model-agnostic and easy to implement and enhances the model’s ability to retain old knowledge and mitigate catastrophic forgetting. Zhang et al. (2023b) propose a de-bias contrastive learning based approach for multimodal named entity recognition (MNER), which studies modality alignment enhanced by cross-modal contrastive learning. Mo et al. (2024) introduce MCL-NER, a multi-view contrastive learning framework for cross-lingual NER that encompasses semantic and token-to-token relation contrastive learning, and constructs code-switched data by randomly replacing some phrases with the target counterparts for the semantic contrastive learning of the source and corresponding code-switched sentence, and performs better than baselines by a large margin on the XTREME-40 and CoNLL benchmarks.

While these methods have significantly advanced the NER task in both general and legal domains, there is a compelling need for a more comprehensive exploration of NER in the legal texts. Existing approaches often overlook the incorporation of semantic information related to entity types. Given the diversity and professionalism of entity types within legal texts, most current methods struggle to accurately assign entity types to entity spans in the legal domain. Additionally, legal texts frequently contain intricate and lengthy entities, posing challenges for both sequence tagging and span-based methods in entity extraction. To address these issues, we have devised a simple sequence-to-sequence framework aimed at enhancing NER performance in the legal domain.

Problem formulation

In our method, the NER task is translated into a sequence to sequence task, which can be formulated as follows: given an sentence of $n$ tokens $X=\{x_1,x_2,...,x_n\}$ and an entity type set $Y=\{y_1,y_2,...,y_r\}$, where $r$ is the number of entity types. We view X as input sequence, and the target sequence is to convert entity spans in the X to a span like “[entity span $\mid$ $y_i$]”, where $y_i\in Y$ and the entity span belongs to $y_i$.

Comprehensive discussion

Seq2seq models are inherently suited for tasks involving complex entity structures, such as those found in legal texts where entities like case references, legal citations, or multi-component names span several words or phrases. These models excel in capturing contextual dependencies due to their attention mechanisms, which significantly improves accuracy in identifying and classifying entities based on the surrounding text. Differentiating between various legal entity types also requires a deep understanding of the domain’s semantics, which seq2seq models address by incorporating entity-type-aware modules that use contrastive learning. This enhances the model’s ability to distinguish between different types of entities effectively. Moreover, seq2seq models address the challenge of predicting entity boundaries by framing entity recognition as a generation task, allowing for more flexible and accurate extraction of nested or overlapping entities. They can also integrate legal-specific knowledge by pre-training on large corpora of legal documents or incorporating legal dictionaries and definitions into the training process. Finally, seq2seq models are adaptable and scalable; they can be fine-tuned with examples from specific areas of law or jurisdictions to handle variability across legal documents and scale well with the addition of more training data.

Encoder optimized by semantic information of entity types

In our method, the sequence to sequence pre-trained model BART is used as our base model. The BART encoder is responsible for encoding the text and captures the features of the text and providing the decoder with the semantic information to generate the entity recognition results. However, due to the diversity and professionalism of entity types in the legal text, it is hard for the model to assign each entity span with the entity type accurately. Therefore, an entity-type-awareness module is designed on the encoder to assist the model in learning the semantic information of entity types and the difference between each entity type, which can optimize the encoding capability of encoder and reduce the entity type prediction errors.

Decoder with copy mechanism

To solve the problem of intricate and lengthy entities in the legal texts and avoid the problem of wrong generation, inspired by See, Liu & Manning (2017), the decoder is utilized to directly generate the entity recognition results, and the copy mechanism is utilized to restrict the process of decoding to ensure that the entity recognition results generated are consistent with corresponding parts of origin texts.

Formally, given an input text $X=\{x_1,x_2,...,x_n\}$, the input text is fed into the encoder to obtain the embedding of the text, and the encoder’s last hidden state is fed into the decoder for the entity recognition results generation. At the $c$-th step of the decoder, the encoder’s last hidden state $h^{enc}$ and the previous output tokens $t_{1:c-1}$ are fed into the decoder, and the last hidden state of the decoder is calculated as follows:

(8) $$h^{enc}=Encoder(X),$$

(9) $$h_c^{dec}=Decoder(h^{enc},t_{1:c-1}),$$

During the training and inference phase, the copy mechanism is utilized to restrict the process of decoding. The attention distribution of token $t_c$ and each token in the input sentence X are first calculated:

(10) $$g_i^c=v^T(W_h{h}_i^{enc}+W_s{h}_c^{dec}+b),$$

(11) $$a^c=softmax(g^c),$$where $v$, $W_h$, $W_s$, $b$ are the learnable parameters. Then the context vector $h_c^{\ast }$ is calculated as follows:

(12) $$h_c^{\ast }=\sum _{i=1}^n{a}_i^c{h}_i^{enc},$$where $h_i^{enc}$ is the encoder hidden state of the token $x_i$, and $a_i^c$ is the attention score of token $t_c$ and token $x_i$. The vocabulary distribution $P_{vocab}^c$ is calculated as follow:

(13) $$P^c=Softmax(Linear(h_c^{dec})),$$

(14) $$P_{gen}^c=Softmax(Linear([h_c^{\ast };h_c^{dec}])),$$

(15) $$P_{vocab}^c=p_{gen}^c{P}^c+(1-p_{gen})\sum _{i=1}^n{a}_i^c,$$where $P_{gen}^c\in [0,1]$ is the weight to choose between generating a word from the vocabulary or copying a word from the input sentence. We can find that the probability of word $w$ is the sum of probability of generating from the vocabulary and copying from the input sentence. If $w$ is an out-of-vocabulary word, $p^c(w)$ will be equal to 0; if $w$ is not in the input sentence, ${\sum }_{i=1}^n{a}_i^c$ will be equal to 0. In the training phase, the loss of decoder $L_{dec}$ at $c$-th step is calculated as follow:

(16) $$L_c^{dec}=-log(P_{vocab}^c(\hat{w})),$$where $\hat{w}$ is the target token at $c-th$ step. Then we calculate the overall loss as follows:

(17) $$L^{dec}=-\frac{1}{G}\sum _{c=1}^G{L}_c^{dec}.$$where G is the length of target sequence. In the inference phase, the greedy search is utilized to decode and the target sequence is generated, which selects the token with the highest probability at step $c$ as the output token.

Train and inference

During the training phase, the loss of contrastive learning task in the entity-type-aware module and the loss of entity recognition results generation task are jointly optimized. The total loss is calculated as follows:

(18) $$L_{total}=\alpha L_{con}+(1-\alpha )L_{dec}.$$where $\alpha$ is the weight between the loss of contrastive learning task and the loss of entity recognition results generation task.

Inspired by Athiwaratkun et al. (2020), during the training and inference phases, given an input text X, the target sequence is considered to convert entity spans in the X to a span like “[entity span $\mid$entity type]”. This approach of constructing the target sequence not only leverages the powerful text reconstruction and contextual understanding capabilities of the BART model, but also provides explicit contextual information during the text generation process, which can reduce the ambiguity in the process of generating text.

During the inference phase, the input text is fed into BART and the greedy search algorithm is utilized to directly generate the target sequence. After obtaining the target sequence, the entity recognition results are obtained by rule post-processing.

Setting

Baselines

To explore the effect of our framework in NER task in the legal domain, we use the following models as baselines.

• BiAffineNER (Yu, Bohnet & Poesio, 2020) utilizes the biaffine mechanism to enhance the ability of model for locating entity spans.

• BERT-MRC (Li et al., 2020) formulates the NER task as a machine reading comprehension task, and extracts possible text spans by answering the queries given the contexts.

• BOCNER (Li et al., 2021) combines the ordered neurons LSTM network (Shen et al., 2018) with the BERT model (Devlin et al., 2019) to tackle the NER task in the legal domain.

• W²NER (Li et al., 2022b) models the NER task as word-word relation classification, and extracts entities by decoding based on the relation between word and word.

• RoBERTa-BiLSTM-CRF (Shi et al., 2022) combines the BiLSTM network with the RoBERTa model to tackle the Chinese NER task in the legal domain.

• T5Based (Lee, Pham & Uzuner, 2022) utilizes the T5 model to tackle the NER task in the financial text in a generative way.

• PUnifiedNER (Lu et al., 2023) is able to jointly train across multiple corpora, and implements in telligent on-demand entity recognition.

• DiffusionNER (Shen et al., 2023) formulates the named entity recognition task as a boundary-denoising diffusion process and thus generates named entities from noisy spans.

Results and analysis

The overall performance of our method compared with the various baselines are shown in Tables 6 and 7. The experiments show that our framework outperforms the previous methods on CAIL2021 and Drug, which demonstrates the effectiveness of our proposed method in the NER task in the legal domain.

Concretely, compared with RoBERTa-BiLSTM-CRF, BOCNER, BiAffineNER, W²NER and PUnifiedNER, our model can achieve better performance on CAIL2021 and Drug. RoBERTa-BiLSTM-CRF, BiaffineNER and BOCNER assist model in capturing the semantic feature of text, which can improve the performance of model in the NER task; owing to the ingenious design of the model, T5Based and W²NER can solve the problem of intricate and lengthy entities in the legal texts, which lead to excellent performance in the NER task in the legal domain; PUnifiedNER is able to prove that the multi-dataset training empowered by prompting can also lead to significant performance gains over single-dataset training. However, they do not take better account of the semantic information of entity types and difference between each entity type, which leads to entity type prediction errors due to the diversity and professionalism of entity types in the legal text. By appropriately integrating entity type information into our model, we enhance its ability to learn semantic details of entity types and the representational differences among them, thereby improving the performance of our model. This approach demonstrates the critical importance of entity type information for NER task in the legal domain.

Compared with BERT-MRC, our model increases by 1.02% and 1.20% in terms of F1 score on CAIL2021 and Drug, respectively. Our model also performs slightly better than DiffusionNER. Similar to our method, BERT-MRC and DiffusionNER encode the comprehensive descriptions of entity type to the query to assist model in learning the information of entity types and improve the effect of model. While BERT-MRC and DiffusionNER incorporate the information of entity types during model training, their decoding process necessitates enumerating all possible entity segments. To maintain efficiency, the length of these segments is often restricted and leads to suboptimal performance in recognizing intricate and lengthy entities within legal texts, thereby compromising the overall effectiveness of the entity recognition task. In contrast, our method incorporates an entity-type-aware module within the encoder, which not only aids in learning the information of entity types and the distinctions among different entity types but also utilizes a decoder equipped with a copy mechanism to directly generate results from entity extraction. This approach enables superior recognition of intricate and lengthy entities in legal texts. These factors help our method outperform BERT-MRC and DiffusionNER in legal NER task.

Cross-validation discussion

In this experiment, we employed a $k$-fold cross-validation technique to enhance the accuracy of model performance assessments. Both the CAIL2021 and Drug datasets were divided into $k=5$ subsets of similar size. For each iteration, one subset was designated as the validation set, while the remaining subsets served as the training set. This process was repeated across five rounds of training and validation. Furthermore, the model was executed five times for each fold (select runs are shown) to secure more robust statistical outcomes. Cross-validation results on the CAIL2021 dataset, the average performance metrics, obtained via cross-validation, are detailed in Table 8.

• Average Precision: $0.935\pm 0.005$

• Average Recall: $0.920\pm 0.004$

• Average F1 Score: $0.927\pm 0.003$

The detailed performance metrics for selected runs and folds are presented in cross-validation results on the Drug dataset. The average performance metrics for the Drug dataset are summarized as follows:

• Average Precision: $0.910\pm 0.006$

• Average Recall: $0.880\pm 0.005$

• Average F1 Score: $0.895\pm 0.004$

The specific metrics for selected runs and folds are illustrated in Table 9. Through comprehensive cross-validation and multiple executions per fold, the model consistently demonstrated stable performance across various data subsets, evidenced by minimal fluctuations in the performance metrics. The standard deviations of the precision, recall, and F1 scores underscore the reliability of our results, indicating robust generalization capabilities. This stability is essential for effectively handling named entity recognition tasks in the legal domain. Further comparisons with baseline models discussed within this article can substantiate the superior performance of our model.

Ablation study

To further demonstrate the effectiveness of our method, we conduct the ablation study in the following three aspects:

The impact of $\alpha$ in the multi-task learning

To explore the impact of different weights $\alpha$ in multi-task learning on the model’s effectiveness, the model is trained with different weights $\alpha$, and the results are shown in Fig. 3.

As is shown in Fig. 3, we find that: (i) When $\alpha$ is equal to 0.05, 0.1 and 0.15 on CAIL2021, and when $\alpha$ is greater than 0.1 on Drug, the effect of model is better than that in the case of $\alpha$ equal to 0. (ii) When $\alpha$ is greater than or equal to 0.2 on CAIL2021, the effect of model is worse than that in the case of $\alpha$ equal to 0. We speculate that when $\alpha$ is set to an appropriate value, the contrastive learning task in the entity-type-aware module can not only assist the model in learning semantic information of entity types and the difference between each entity type, but also will not negatively influence the understanding and generation ability of the model, which makes the effect of the model have an overall improvement. On CAIL2021, when $\alpha$ is too large, the model assigns too much weight to the contrastive learning task during the training process, which negatively affects the optimization of the entity recognition results generation task, and influences the model’s ability to understand the semantic information and generate the text. Consequently, it leads to the performance degradation of the model. The experimental results on Drug show that when the $\alpha$ value is low, the model may be too biased towards the entity recognition results generation task, and ignores the information gain that may be brought by the contrastive learning task, resulting in the F1 value not being significantly improved; while when the $\alpha$ value is gradually increased from 0.15 to 0.25, the model is able to reasonably balance the entity recognition results generation task and the contrastive learning task, and make full use of the two complementary advantages to achieve performance improvement. However, too high $\alpha$ values (e.g., when $\alpha$ exceeds 0.25) may cause the model to consider the contrastive learning task too much and sacrifice the performance of the entity recognition results generation task, resulting in a fluctuating trend of high and low F1 values. As is shown in Fig. 3, the model achieves the optimal performance on the CAIL2021 when $\alpha$ is set to 0.1, whereas setting $\alpha$ to 0.25 yields the best performance on the Drug dataset.

Temperature coefficient $\tau$ in the contrastive learning task

Temperature coefficient $\tau$ is a important parameter in the contrastive learning task, because it can affect the shape of the logits distribution and model’s discrimination against negative samples. To explore the influence of different temperature coefficient $\tau$ in the contrastive learning task on the effect of the model, we train the model with different temperature coefficient $\tau$, and the results on CAIL2021 and Drug are shown in Fig. 4.

It is obvious that when $\tau$ is set to a smaller value, the model’s performance is significantly better than that when $\tau$ is set to a larger value. A possible reason is that a smaller temperature coefficient makes the model understand and utilize hard negative samples more effectively, so it is easier to form a uniform representation space, which leads to better performance of the model. Therefore, the performance of the model is greatly influenced by the setting of the temperature coefficient, highlighting the importance of setting an appropriate coefficient for each dataset.

Learning rate in hyperparameters of the BART model

In the BART model, the learning rate parameter is critical to model training. The learning rate determines how much the model weights are adjusted, i.e., how quickly the model responds to the loss function in each iteration. To assess the impact of the learning rate on model performance, the model is trained with different learning rates, and the results on CAIL2021 and Drug are shown in Fig. 5.

As shown in Fig. 5, on the CAIL2021 dataset, the optimal learning rate that achieves the best model performance is 5e-5, at which the F1 score peaks. When the learning rate is below this value, the model’s F1 score progressively increases, suggesting that a lower learning rate enhances the model’s ability to capture intricate details in legal texts. Conversely, when the learning rate exceeds 5e-5, the F1 score gradually decreases, which suggests that a high learning rate might cause oscillations during model training, adversely affecting convergence and performance. On the Drug dataset, the optimal learning rate is 3e-5. This dataset exhibits higher sensitivity to learning rate changes; a slight increase to 3e-5 significantly reduces performance, demonstrating a strict need for precise learning rate selection. The experimental results show that the moderate learning rate can better balance the model training speed and performance, and avoid the problem of overfitting and underfitting.

Batch size in hyperparameters of the BART model

Batch size, referring to the number of samples processed per training iteration, directly influences the gradient estimation and memory usage of the BART model. An optimal batch size ensures more stable gradient estimations, and facilitates faster model convergence while efficiently leveraging the parallel processing capabilities of modern hardware. To assess the impact of batch size on model performance, the experiments are conducted across different batch sizes, with the results illustrated in Fig. 6.

According to the results in Fig. 6, on the CAIL2021 dataset, the F1 score initially increases with the batch size but starts to decline after reaching a peak at a batch size of 12, demonstrating optimal data feature capture and generalization at this size. However, as the batch size increases to 24, the F1 score gradually decreases, possibly due to reduced exposure to diverse data per training cycle, which could hinder the model’s ability to learn from a broader data distribution. Similarly, the Drug dataset shows an increase in F1 score with larger batch size, peaking at 12 before beginning to decrease. The above results suggest that while the smaller batch size can increase model update frequency, it may lead to the larger fluctuation during training. Conversely, overly large batch size might impair the model’s generalization capability and training efficiency. Therefore, appropriately configuring batch size can accelerate model convergence while maintaining balance between generalization performance and training stability, thereby achieving the optimal outcomes in legal domain.

Top- $\mathit{p}$ in hyperparameters of the BART model

When using the BART model for named entity recognition, the top- $p$ parameter is a crucial factor that governs diversity in generation by determining which words are considered as potential outputs once the cumulative probability reaches a certain threshold. An array of top- $p$ settings ranging from 0.50 to 0.85 are tested on CAIL2021 and Drug. Detailed experimental results are shown in Fig. 7.

According to the results shown in Fig. 7, the F1 score on the CAIL2021 dataset initially increases with top- $p$ and reaches the optimal performance at a top- $p$ of 0.70. Beyond this point, as top- $p$ continues to increase, the F1 score begins to decrease, possibly due to the introduction of excessive irrelevant information, which reduces the model’s precision. Similarly, the Drug dataset shows a peak F1 score at a top- $p$ of 0.65, with performance declining thereafter due to noise. The above results suggest that the selection of top- $p$ should be optimized based on the specific characteristics of the dataset; such a low top- $p$ may limit the model’s learning capacity, while such a high value can lead to information overload, affecting the model’s accuracy and generalizability.

Dropout rate in hyperparameters of the BART model

Dropout, as an effective regularization technique, helps to mitigate overfitting and enhance model generalization on unseen data. By conducting tests with eight different dropout rates on the CAIL2021 and Drug datasets, the specific impact on the F1 score is observed. The detailed experimental results are shown in Fig. 8.

According to the results shown in Fig. 8, the F1 score on the CAIL2021 dataset initially increases with the rise in dropout rate, peaking at 0.15, before declining with further increases. Similarly, on the Drug dataset, the highest F1 score occurs at a dropout rate of 0.10, with performance dropping thereafter. The above results suggest that the optimal dropout rate can help judicial named entity recognition models balance model complexity and noise handling, preventing overfitting. However, such a low dropout rate might not sufficiently deactivate neurons randomly during training, potentially leading to overfitting, while such a high dropout rate could lead to excessive information loss, impairing the model’s ability to capture key features.

Analysis of intricate and lengthy entity recognition

To illustrate the performance of the model on intricate and lengthy entities, the entities are divided into three groups according to their lengths. Our model is compared with W²NER, which models the NER task as word-word relation classification and enables intricate and lengthy entity recognition. Tables 14 and 15 show the experimental results. Compare with W²NER, our model increases by 0.31% and 5.35% in term of F1 for intricate and lengthy entities ( $L>10$) on CAIL2021 and Drug, respectively. A possible explanation is that W²NER extracts entities by decoding based on the relation between word and word, which may lead to error propagation problem and affect its effect of recognizing intricate and lengthy entities. The better performance of our model demonstrates that our model is more effective in recognizing intricate and lengthy entities.

Analysis of the scalability of the BART model

In the ideal scenario, as data scale increases, the model’s inference time should grow linearly or even less. If experimental results demonstrate that processing time increases more slowly than data scale, this indicates good scalability of the model. To more accurately assess the scalability of the BART model, we design experiments with datasets of varying sizes from small to large scale. We then conduct inference using the BART model on these datasets, and the variation of inference time and F1 score with token size is shown in Figs. 9 and 10, respectively.

From the results depicted in Figs. 9 and 10, it is observable that on the CAIL2021 and Drug datasets, inference time increases with the number of tokens across eight different dataset scales, but not strictly linearly. The rate of increase in inference time is slightly slower than the rate of token increase, indicating that the BART model scales well with large datasets. For the performance variation, as data scales up, the model’s F1 score slightly improves initially and then begins to decline slightly. This trend might reflect the model’s enhanced learning and generalization capabilities on medium-sized datasets, but as the dataset size increases, so do the challenges, possibly causing a slight performance decline. This performance dip could be also related to increased noise in larger datasets, changes in entity distribution, or more complex contextual relationships. Hence, the BART model demonstrates good scalability and effective handling of varying sizes of judicial named entity recognition datasets. Despite the increase in inference time with larger data scales, the sub-linear rate of time growth shows efficient large dataset handling, and signifies robust scalability. Performance variations across different data sizes show some fluctuations but remain relatively stable, suggesting that the BART model provides consistently stable performance, even on large-scale datasets.

Case study

Some cases are illustrated in Table 16 to demonstrate that entity type prediction errors can be effectively reduced, and intricate and lengthy entities in legal texts can be accurately extracted by our model. In case 1, with the assistance of an entity-type-aware module to learn rich semantic information about entity types and the differences between each entity type, the entity type can be correctly assigned for the entity span by our model. In case 2, intricate and lengthy entities in the legal text can be accurately extracted by our model through the generation-based extraction method with the copy mechanism.