A literature survey of shapelet quality measures for time series classification

Searching

Initially, we formulated comprehensive query terms to retrieve the maximum relevant literature. For this survey, the following databases were systematically searched: Google Scholar, IEEE Xplore Digital Library, ACM Digital Library, Scopus, and Web of Science. These platforms collectively encompass nearly all major computer science engineering research domains through their interdisciplinary indexing and extensive publication coverage. Considering the theme of this survey, the search keywords of query are designed as: “shapelet” AND “time series classification” AND “quality” AND (“evaluate” OR “evaluation” OR “assess” OR “assessment” OR “measurement” OR “measure”). The number of papers fetched from the selected databases are shown in Fig. 3.

Filtering

Two criteria were used to filter out some irrelevant papers:

• 1.

  Duplicate papers only require the formal final version. As shown in Fig. 3, 194 papers were filtered out according to this criteria.

• 2.

  The language of the paper must be in English. Papers with other languages will be filter out. As shown in Fig. 3, four papers were filter out according to this criteria.

Selection

Finally, we selected the final paper which related shapelet quality measures for time series classification. At first, we selected the 1,036 officially published papers from the 1,224 papers. An officially published paper must be a journal paper, a conference paper, or a book chapter, and it is not retracted. Following by, we have read the full-text of the 1,036 officially published papers. After filtering out 943 papers that only mentioned the names of measures, only 93 papers that formally described methods for evaluating shapelet quality were ultimately selected. Finally, we identified 20 shapelet quality methods from these 93 papers.

Information gain

Information gain is a widely recognized quality measure for candidate shapelets and has proven to be highly competitive for shapelet-based time series classification methods (Guijo-Rubio et al., 2020). When shapelets were first proposed (Ye & Keogh, 2009), information gain (Shannon, 1948) was adopted as the quality measure. Information gain is calculated based on entropy. For a given dataset D, the entropy e(D) can be computed using the equation:

(2) $$e(D)=-\sum _{i=1}^C{p}_i\log p_i,$$where C represents the number of classes in the dataset, and $p_i$ is is the probability that the time series belongs to class i.

A candidate shapelet S with a distance threshold $d_{\epsilon }$ can separate the training dataset into two non-overlapping subsets, $D_L$ and $D_R$. Time series in D that are farther from S than $d_{\epsilon }$ are placed in $D_R$; otherwise, they are assigned to $D_L$. After this separation, the information gain IG can be calculated as follows:

(3) $$IG=e(D)-\frac{|D_L|}{|D|}e(D_L)-\frac{|D_R|}{|D|}e(D_R),$$where e(D), $e(D_L)$, and $e(D_R)$ represent the entropy of the respective datasets, and |D|, $|D_L|$, and $|D_R|$ denote the number of time series contained in each corresponding dataset.

At the beginning, the information gains of all candidate shapelets were compared collectively. However, this approach may lead to certain classes being underrepresented. Therefore, researchers select shapelets for each class proportionally, rather than relying solely on the magnitude of information gain (Bostrom & Bagnall, 2015; Le et al., 2024). To address over-fitted shapelets, Kramakum, Rakthanmanon & Waiyamai (2018) aggregated information gain of the shapelet candidate with its surrounding. The information gain value is usually similar for an unbalanced dataset (Chen, Fang & Wang, 2024). For this, Chen, Fang & Wang (2024) adopted the information gain ratio to assess the shapelet candidates. The information gain ratio can be obtained as

(4) $$IG=1-\frac{\frac{|D_L|}{|D|}e(D_L)-\frac{|D_R|}{|D|}e(D_R)}{e(D)}$$where e(D), $e(D_L)$, $e(D_R)$, |D|, $|D_L|$, and $|D_R|$ have the same meanings as them in Eq. (3).

Using information gain to evaluate candidate shapelets is very time-consuming (Bostrom & Bagnall, 2017). One reason for this is that the number of candidate shapelets is massive, as any subsequence can be considered a candidate shapelet. Thus, the quality evaluation process can be accelerated by applying various filtering techniques, such as length limits (Hills et al., 2014; Yamaguchi, Ueno & Kashima, 2023), Symbolic Aggregate approXimation (SAX) (Rakthanmanon & Keogh, 2013; Li et al., 2020), key points (Li, Yan & Wu, 2019; Li et al., 2023), and random sampling (Karlsson, Papapetrou & Boström, 2016; Ji et al., 2023). Another reason for the high computational cost is that evaluating a single candidate shapelet involves significant complexity. To address this, some researchers have accelerated the quality evaluation process by reducing measurement complexity through techniques like subsequence distance early abandonment (Ye & Keogh, 2011), admissible entropy pruning (Ye & Keogh, 2011), and reusing computations (Xing et al., 2011; Mueen, Keogh & Young, 2011; Wan et al., 2023). Additionally, some researchers have proposed faster evaluation measures.

F-statistic

At the same time as proposing the shapelet transformation, Lines et al. (2012) used the F-statistic (Mood, 1950) as an alternative quality measure for candidate shapelets. There are three main steps to calculate the F-statistic:

• 1.

  By calculating the distances between each time series in the dataset and the candidate shapelet S, a list of distances $L=⟨d_1,d_2,\cdots ,d_n⟩$ is obtained.

• 2.

  Separate L into several lists $L_1,\dots ,L_i,\dots ,L_C$ based on class membership, such that the distances between all time series that belong to class i are stored in $L_i$.

• 3.

  Calculate the F-statistic quality measure as (5) $$F=\frac{{\sum }_{i=1}^C{(\overline{L_i}-\overline{L})}^2/(C-1)}{{\sum }_{i=1}^C{\sum }_{d_j\in L_i}{(d_j-\overline{L_i})}^2/(n\text{-}C)}$$where C is the number of classes, n is the total number of time series, $\overline{L_i}$ is the average of the values in the list $L_i$, and $\overline{L}$ is the overall average of the values in the entire list L.

In this method, the larger the F value, the better the quality of the candidate shapelet.

Kruskal-Wallis

Lines & Bagnall (2012) first used the Kruskal-Wallis test (Kruskal, 1952) to evaluate the quality of candidate shapelets. As a non-parametric test, the Kruskal-Wallis test assesses the quality of candidate shapelets based on the distribution of distances between each time series and the candidate shapelet S. The procedure to obtain the Kruskal-Wallis statistic involves four main steps:

• 1.

  First, calculate the distances between each time series and S.

• 2.

  Next, assign a rank value to each distance, ordering them from smallest to largest.

• 3.

  Then, split the ranks by class membership into sets $R_1,\dots ,R_C$.

• 4.

  Finally, the Kruskal-Wallis statistic for S can be calculated as follows: (6) $$KW=\frac{12}{n\cdot (n+1)}{\sum }_{i=1}^C\frac{{\sum }_{r_j\in R_i}{r_j}^2}{|R_i|}-3(n+1)$$where n is the total number of time series, C is the number of classes, $R_i$ represents the ranks for class i, and $|R_i|$ is the number of time series in class i.

Mood’s median

Lines & Bagnall (2012) first adopted Mood’s median for shapelets. This is another non-parametric test used to assess the quality of candidate shapelets. The procedure to obtain Mood’s median statistic consists of four main steps:

• 1.

  Calculate the distances between each time series and the candidate shapelet S, then obtain their median $d_m$.

• 2.

  For each class i, record $o_{i1}$ (the number of distances above $d_m$) and $o_{i2}$ (the number of distances below $d_m$).

• 3.

  For each class i, calculate $e_{i1}$ (the expected number of distances above the median) and $e_{i2}$ (the expected number of distances below the median) as follows: (7) $$e_{i1}=p_i\cdot \sum _{k=1}^C{o}_{k1}$$and (8) $$e_{i2}=p_i\cdot \sum _{k=1}^C{o}_{k2}$$where C is the number of classes, and $p_i$ is the probability of class i.

• 4.

  According to Hills et al. (2014), the final Mood’s median statistic of S can be obtained as: (9) $$M=\sum _{i=1}^C\sum _{j=1}^2\frac{{(o_{ij}-e_{ij})}^2}{e_{ij}}.$$

F-measure

The F-measure (which is also be called Utility in some paper) was adopted by Xing et al. (2011) to assess the shapelet quality. There are four steps to obtain the F-measure of the shapelet candidate S.

• 1.

  Obtain the distances between each time series T and S.

• 2.

  Calculate the precision of S with the threshold $\delta$ as (10) $$Precision=\frac{N_{d<\delta \wedge L(S)}}{N_{d<\delta }}.$$where $N_{d<\delta }$ is the number of time series with a distance less than $\delta$ from S, $N_{d<\delta }\wedge L(S)$ is the number of time series which have the class label with a distance less than $\delta$ from S and in the same class as T.

• 3.

  Calculate the recall of S with the threshold $\delta$ as (11) $$Recall=\frac{N_{d<\delta \wedge L(S)}}{N_S}.$$where $N_{d<\delta }$ is the number of time series in the same class as T, $N_{d<\delta }\wedge L(S)$ is the number of time series which have the class label with a distance less than $\delta$ from S and in the same class as T.

• 4.

  Get the final F-measure of S as (12) $$F=\frac{2\cdot Recall\cdot Precision}{Recall+Precision}.$$

Silhouette score

Zhu, Lu & Sun (2015) adopted the Silhouette score (Rousseeuw, 1987) as the clustering indicator and reduced the number of candidate shapelets. Later, the Silhouette score was directly used as a measure to assess the quality of shapelets by Charane, Ceccarello & Gamper (2024). In their work, the Silhouette score is calculated based on whether a time series has the same class label as that of the candidate shapelet or not. The calculation of the Silhouette score for one candidate shapelet S involves mainly two steps:

• 1.

  Calculate the distances between each time series T and S.

• 2.

  The final Silhouette score of S can be obtained as (13) $$S=\frac{\frac{{\sum }_{T\notin D_c}d(T,S)}{|D|-|D_c|}-\frac{{\sum }_{T\in D_c}d(T,S)}{|D_c|}}{max\left(\frac{{\sum }_{T\notin D_c}d(T,S)}{|D|-|D_c|},\frac{{\sum }_{T\in D_c}d(T,S)}{|D_c|}\right)}$$where c is the class label of the time series that contains S, $d(T,S)$ is the distance between T and S, $D_c$ represents the time series from the same class c, D contains all the time series, $|D_c|$ is the number of time series in $D_c$, |D| is the number of time series in D, and $max$ is used to obtain the maximum value.

Cumulative distance

Liu et al. (2016) selected shapelets for each class through the cumulative distance separately. For class i, the cumulative distance of shapelet candidate S can be calculated as

(14) $$cd={\sum }_{T\in D_i}d(T,S),$$where $D_i$ is the set of all time series belonging to class i, $d(T,S)$ is the distance between T and S. Finally, the candidate with the smallest cumulative distance is selected as the shapelet.

Importance measure

Unlike the aforementioned measures of evaluating candidates one by one, Ji et al. (2022) utilize an importance measure to evaluate all candidate shapelets simultaneously with the assistance of a random forest. The following six main steps outline how to obtain the importance measure:

• 1.

  Calculate the distance between each time series and each candidate shapelet.

• 2.

  Construct a random forest based on the calculated distances. In this process, the candidate shapelets are used as features, while the corresponding distances serve as feature values.

• 3.

  Calculate the Gini impurity of each node in the random forest using the formula (15) $$GI=1\text{-}{\sum }_{i=1}^C{p_i}^2,$$where C is the number of classes and $p_i$ is the probability of class i.

• 4.

  For each decision node, obtain the importance measure as (16) $$IM=GI\text{-}G{I}_l-G{I}_r,$$where GI, $G{I}_l$, and $G{I}_r$ represent the Gini impurity of the current node, the left sub-node, and the right sub-node, respectively.

• 5.

  The importance measure of the candidate shapelet $IM(S)$ is the sum of the importance measures of the nodes that use S as a decision condition.

• 6.

  Finally, normalize the importance measures.

Area under curve

Addressing the class imbalance problem, Yan & Cao (2020) adopted the AUC value to evaluate the quality of candidate shapelets in binary-class problems. The calculation of the AUC value for a candidate shapelet S involves three main steps:

• 1.

  Calculate the distance between each time series and the candidate shapelet S.

• 2.

  Assign a rank value to the distances, ordering them from smallest to largest.

• 3.

  Obtain the AUC value of the candidate shapelet as follows: (17) $$AUC=\frac{{\sum }_{T_i\in D_P}{r}_i-\frac{|D_P|\ast (|D_P|+1)}{2}}{|D_P|\ast |D_N|}$$where $D_P$ contains all the positive time series, $|D_P|$ is the number of positive instances, $|D_N|$ is the number of negative instances, and $r_i$ is the rank value of the time series $T_i$.

Distance measure

Chen & Wan (2023) designed a distance measure to evaluate candidate shapelets. Their distance measure includes four steps:

• 1.

  Generate a subsequence set for each time series $T_i$ through a sliding window of length L, and record the set as $set(T_i)=⟨S_1^i,\cdots ,S_j^i,\cdots ,S_{|T_i|-L+1}^i⟩$, where $|T_i|$ is the length of $T_i$, and $S_j^i$ is a continuous subsequence starting at position j.

• 2.

  For each subsequence $S_j^i$, calculate the interclass average distance $d_{inter}$ between it and the time series in other categories as (18) $$d_{inter}=\frac{{\sum }_{T_k\in D^{-}}d(S_j^i,T_k)}{|D^{-}|},$$where $D^{-}$ is the dataset that contains all time series not belonging to the same class as $T_i$, and $|D^{-}|$ is the number of such time series. In Eq. (18), the term $d(S_j^i,T_k)$ represents the distance between $S_j^i$ and the time series $T_k$. This distance can be calculated as (19) $$d(S_j^i,T_k)=\underset{S_h^k\in set(T_k)}{min}d(S_j^i,S_h^k),$$where $d(S_j^i,S_h^k)$ is the Euclidean distance between the subsequence $S_j^i$ and the subsequence $S_h^k$.

• 3.

  Using a similar method, calculate the intraclass average distance $d_{intra}$ between $S_j^i$ and the time series in the same category as (20) $$d_{intra}=\frac{{\sum }_{T_k\in D^{+}}d(S_j^i,T_k)}{|D^{+}|},$$where $D^{+}$ is the dataset that contains all time series belonging to the same class as $T_i$, and $|D^{+}|$ is the number of such time series.

• 4.

  Based on the interclass average distance $d_{inter}$ and the intraclass average distance $d_{intra}$, the overall distance measure can be obtained as (21) $$dm=\frac{d_{inter}}{d_{intra}}.$$

Location measure

Chen & Wan (2023) believe that the shapelets should meet the following two criteria: (1) the best matching subsequences from the same class are near their location; and (2) the best matching subsequences from different classes are far from their location. Based on this, they designed a location measure to assess the candidate shapelets. Their distance measure includes four steps:

• 1.

  Generate a subsequence set for each time series $T_i$ through a sliding window of length L, and record the set as $set(T_i)=⟨S_1^i,\cdots ,S_j^i,\cdots ,S_{|T_i|-L+1}^i⟩$, where $|T_i|$ is the length of $T_i$, and $S_j^i$ is a continuous subsequence starting at position j.

• 2.

  For each subsequence $S_j^i$, calculate the interclass average location difference $p_{inter}$ between it and the best matching subsequences in other categories as (22) $$p_{inter}=\frac{{\sum }_{T_k\in D^{-}}|p(S_j^i,T_k)-j|}{|D^{-}|},$$where $D^{-}$ is the dataset that contains all time series which are not of the same class as $T_i$, and $|D^{-}|$ is the number of such time series. In Eq. (22), $p(S_j^i,T_k)$ is the position of the best subsequence of $T_k$. The best subsequence of $T_k$ is defined as the subsequence that has the smallest Euclidean distance to $S_j^i$.

• 3.

  Using a similar method, calculate the intraclass average location difference $p_{intra}$ between $S_j^i$ and time series in the same category as (23) $$p_{intra}=\frac{{\sum }_{T_k\in D^{+}}|p(S_j^i,T_k)-j|}{|D^{+}|},$$where $D^{+}$ is the dataset that contains all time series which are in the same class as $T_i$, and $|D^{+}|$ is the number of such time series.

• 4.

  Based on the interclass average location difference $p_{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}}$ and the intraclass average location difference $p_{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{a}}$, the location measure can be obtained as (24) $$pm=\frac{p_{inter}}{p_{intra}}.$$

Chi-square statistic

Bock et al. (2018) adopted the Chi-square ${\chi }^2$ statistic to measure shapelet candidates. There are four steps to get the quality of shapelet candidate (S) with the threshold ( $\delta$):

• 1.

  Calculate the distances between each time series T and S.

• 2.

  Fill in the corresponding values in the contingency table for r Chi-square statistic according the class label and distance. The contingency table for Chi-square statistic is shown as Table 1.

• 3.

  Calculating the Chi-square statistic using the value in Table 1. The Chi-square statistic is defined as (25) $$T_{{\chi }^2}=\frac{N{(ac\text{-}bd)}^2}{(a+b)(a+b)(a+d)(b+d)}$$ where a, b, c, and d can be get from Table 1.

• 4.

  Obtain the quality of S as (26) $$quality=1\text{-}{F}_{{\chi }^2}(T_{{\chi }^2})$$where $T_{{\chi }^2}$ can be get for Eq. (25), $F_{{\chi }^2}(\cdot )$ is the cumulative density function of a ${\chi }^2$-distribution.

Gap

Gap is first adopted to assess the quality of shapelet candidates in scenarios without class labels (Ulanova, Begum & Keogh, 2015). It is also suitable for binary classification. The gap can be calculated as

(27) $$gap=({\mu }_{+}+{\sigma }_{+})-({\mu }_{-}+{\sigma }_{-})$$where ${\mu }_{+}$ and ${\sigma }_{+}$ denote the average and standard deviation of the distances between all time series in the positive class and the shapelet candidates, ${\mu }_{-}$ and ${\sigma }_{-}$ denote the average and standard deviation of the distances between all time series in the negative class and the shapelet candidates.

Derivative of information gain

The common measures evaluate how well one candidate shapelet splits up all the classes (Bostrom & Bagnall, 2015). However, the selection of shapelets performed by them may not separate a single class. This phenomenon becomes more severe as the number of classes increases. For this, Bostrom & Bagnall (2015) used one vs. all encoding scheme to transform the multi-class into C (C is the class number) binary-class problem. And then, they adopted the derivative of information gain to evaluate the quality of candidate shapelets. The process of obtaining the derivative of information gain is basically the same as the process of obtaining information gain. The difference between them is the calculation method of entropy. In derivative of information gain, the entropy is calculated as

(28) $$e(D)=-p_c\log p_c-(1-p_c)\log (1-p_c)$$where c is the class label of the time series which the candidate shapelet originated from, and $p_c$ is the probability of class c.

Coverage ratio

Based on the clustering technique, Yang & Jing (2024) proposed the coverage ratio to asses the shapelet quality. There are mainly four steps to obtain the coverage ratio of one candidate shapelet S:

• 1.

  Calculate the distances between each time series T and S.

• 2.

  Cluster the distance into C (which is the number of classes) groups, and numbered in ascending order of distance values. Thus, we obtained C groups ( $G_1$, $G_2$, $\cdots$, $G_C$) with distances from small to large.

• 3.

  Count the number of time series with the same label as S in each group, and record them as $N_1$, $N_2$, $\cdots$, $N_C$).

• 4.

  The final coverage ratio of S is calculated as (29) $$CR=\frac{N_1}{|D_{L(S)}|}-\lambda \ast \frac{{\sum }_{i=2}^C{N}_i}{|D|-|D_{L(S)}|}$$where |D| is the number of all time series, $|D_{L(S)}|$ is the number of time series which has the same label as S, and $\lambda \in (0,1]$ is the weight coefficient.

Combination of F1-score and AUC

Based on the one-vs-one encoding scheme, a multi-class classification problem can be converted into several binary-class problems (Bostrom & Bagnall, 2015). However, directly using binary-class measures, such as AUC, is not suitable for effectively determining the minority class. To address this issue, Yan & Cao (2020) combined the F1-score and AUC to evaluate the quality of candidate shapelets in the multi-class context, as shown in Eq. (30). In Eq. (30), $F1$ represents the F1-score, which can be calculated using Eq. (12);AUC can be obtained from Eq. (17); and IR refers to the imbalance ratio, computed as per Eq. (31). In Eq. (31), C denotes the number of classes, $n$ is the total number of time series, and $n_i$ is the number of time series in class $i$.

(30) $$score=\frac{1}{IR}\cdot F1\text{-}\left(1-\frac{1}{IR}\right)\cdot AUC$$

(31) $$IR=\frac{1}{C}\sum _{i=1}^C\frac{n-n_i}{C\cdot n_i}.$$

Ordinal Fisher score

The Fisher score (Hart, Stork & Wiley, 2001) has been utilized for feature selection by several researchers (Gu, Li & Han, 2011). By reformulating it, Pérez-Ortiz et al. (2016) introduced the ordinal Fisher score to address the challenges of ordinal classification problems. Subsequently, Guijo-Rubio et al. (2020) applied the ordinal Fisher score to evaluate shapelet quality for ordinal time series classification problems. The process of obtaining the ordinal Fisher score involves four main steps:

• 1.

  Calculate the distances between each time series in the dataset and the candidate shapelet S, resulting in a list of distances $L=⟨d_1,d_2,\dots ,d_n⟩$.

• 2.

  Separate L into several lists $L_1,\dots ,L_i,\dots ,L_C$ based on class membership, such that the distances corresponding to class i are grouped into $L_i$.

• 3.

  For each list $L_i$, compute its mean $\overline{L_i}$ and its standard deviation $S_i$.

• 4.

  Obtain the ordinal Fisher score as follows: (32) $$OFS=\frac{{\sum }_{i=1}^C{\sum }_{j=1}^C|i\text{-}j|{(\overline{L_i}-\overline{L_j})}^2}{(C-1){\sum }_{i=1}^C{S_i}^2}$$where C is the number of classes. In Eq. (32), $|i\text{-}j|$ serves to penalize the distances between farther classes.

Pearson’s correlation coefficient

Guijo-Rubio et al. (2020) adopted the Pearson’s correlation coefficient as the quality measure for ordinal classification. They calculated the Pearson’s correlation coefficient based on the distances obtained from the shapelet and the class labels. The Pearson’s correlation coefficient was primarily obtained through the following three steps:

• 1.

  Calculate the distances between each time series and the candidate shapelet S, recording them as $L=⟨d_1,d_2,\dots ,d_n⟩$.

• 2.

  Calculate the difference between the class of each time series and S, recording them as $LC=⟨d{c}_1,d{c}_2,\dots ,d{c}_n⟩$. For a given time series $T_i$ and S, $d{c}_i$ can be obtained as (33) $$d{c}_i=|P_{L(T_i)}-P_{L(S)}|$$where $L(T_i)$ is the class label of $T_i$, L(S) is the class label of S, and $P_{\ast }$ denotes the position of the class $\ast$ in the ordinal scale.

• 3.

  Finally, obtain the Pearson’s correlation coefficient as (34) $$PCC=\frac{cov(L,LC)}{S_L\cdot S_{LC}}$$where $cov(L,LC)$ is the covariance between L and LC, $S_L$ is the standard deviation of L, and $S_{LC}$ is the standard deviation of LC.

Spearman’s correlation coefficient

Spearman’s correlation coefficient was introduced by Guijo-Rubio et al. (2020) to assess the quality of the candidate shapelets. They mainly obtained Spearman’s correlation coefficient through the following five steps:

• 1.

  The distances between each time series and the candidate shapelet S were calculated and recorded as $L=⟨d_1,\cdots ,d_i,\cdots ,d_n⟩$.

• 2.

  Each distance $d_i$ is assigned a rank value $R_i$ from smallest to largest.

• 3.

  The difference between the class of each time series and S was calculated and recorded as $LC=⟨d{c}_1,\cdots ,d{c}_i,\cdots ,d{c}_n⟩$. Here, $d{c}_i$ is calculated according to Eq. (33).

• 4.

  Each difference $d{c}_i$ is assigned a rank value $R{c}_i$ from smallest to largest.

• 5.

  Finally, the Spearman’s correlation coefficient is calculated as (35) $$SCC=1-\frac{6{\sum }_{j=1}^C{(R_i-R{c}_i)}^2}{n(n^2-1)}$$where n is the number of time series.

Concentration and dominance of coverage

Jing & Yang (2024) proposed the CD-Cover as a quality measure for ordinal classification problems. Unlike most methods that utilize real subsequences, they ultimately selected some SAX (Lin et al., 2007) sequences as the final shapelets. The CD-Cover of candidate shapelet S can be obtained through the following six steps:

• 1.

  Represent all time series in the SAX format.

• 2.

  Count the coverage of S and denote it as $\varphi =⟨{\lambda }_1,\cdots ,{\lambda }_i,\cdots ,{\lambda }_C⟩$, where C is the number of classes and ${\lambda }_i$ is the number of time series containing S in class i.

• 3.

  Calculate the concentration of coverage of S as: (36) $$con=1-\frac{var(\varphi )}{{(C-1)}^2/4}$$where $var(\varphi )$ is the variance of the coverage $\varphi$.

• 4.

  Compute the coverage rate of S, denoting it as $\pi =⟨{\kappa }_1,\cdots ,{\kappa }_i,\cdots ,{\kappa }_C⟩$, where ${\kappa }_i$ is calculated as ${\lambda }_i/n_i$ ( $n_i$ is the number of time series in class i).

• 5.

  Calculate the dominance of coverage as: (37) $$dom=\underset{1}{max}(\pi )-\underset{2}{max}(\pi )$$where $\underset{1}{max}(\pi )$ and $\underset{2}{max}(\pi )$ represent the largest and second-largest values in $\pi$, respectively.

• 6.

  Obtain the CD-Cover of S as: (38) $$\sigma =\alpha \cdot con+(1\text{-}\alpha )\cdot dom$$where $\alpha \in (0,1)$ is a weight coefficient; in general, $\alpha$ is set to 0.5.

Challenges

After a thorough review of shapelet quality measures, we identify the following points: shapelets selected through these measures can accurately classify time series data and provide a degree of interpretability. Despite the promising results, several challenges remain:

• Excessive dependence on Euclidean distance. With the exception of CD-Cover, all measures necessitate the computation of distances between candidate shapelets and time series data. The predominant reliance on Euclidean distance in the existing literature (Costa et al., 2020; Amouri et al., 2022) raises concerns about its limitations in capturing complex relationships within diverse datasets. This dependence may compromise performance when the underlying data distributions deviate from the assumptions inherent in Euclidean metrics.

• High time complexity. Many shapelet quality measures exhibit high computational intensity (Wan et al., 2023). This complexity arises not only from the extensive number of candidate shapelets (Ji et al., 2019) but also from the inherently time-consuming nature of the measures themselves (Ye & Keogh, 2011; Mbouopda & Nguifo, 2024). The trade-off between rigorous evaluation and practical applicability becomes evident; while thorough assessments are essential for robust model performance, they can significantly hinder scalability and real-time processing capabilities.

• Lack of consideration for interpretability. One notable advantage of shapelets is their intuitive interpretability (Chen & Wan, 2023). However, current measures tend to emphasize categorization ability at the expense of comprehensibility for end-users. This oversight presents a critical trade-off: while performance metrics may indicate efficacy in classification tasks, they often neglect the importance of presenting understandable results that can be effectively communicated to users. Balancing interpretability with performance remains a vital consideration for advancing shapelet-based methods.

Future research directions

In response to the above challenges, combined with current research hotspots, several promising areas for further research have been identified, as discussed below:

• Revolutionizing measure efficiency for next-generation challenges. As datasets scale toward exabyte volumes and real-time analytics demands intensify, future research must transcend hardware-specific optimizations. We envision algorithmic-level breakthroughs: measures leveraging quantum-inspired annealing for hyper-fast candidate screening, or federated evaluation schemes preserving privacy in distributed environments. Simultaneously, rethinking distance-free quality assessment—potentially through topological invariants or symbolic representations—could bypass computational bottlenecks entirely.

• Embedding human-centric interpretability into quality assessment. The quest for intuitive shapelets demands deeper collaboration between ML and human-computer interaction research. Future measures could incorporate neurosymbolic frameworks, where quality scores reflect both statistical robustness and alignment with domain experts’ mental models. Integrating knowledge graphs for semantic validation, or designing generative adversarial networksto synthesize “interpretability audits”, might bridge the gap between technical efficacy and actionable insights.

• Convergence of shapelets and deep learning architectures. With shapelets evolving into trainable convolutional filters, quality measures must advance beyond traditional criteria. Future work should develop latent-space quality metrics that evaluate shapelets’ contributions to model disentanglement, adversarial robustness, or multi-scale feature coherence. Exploring connections to information bottleneck theory could provide theoretical foundations for “learnable shapelet” assessment, positioning shapelets as interpretable building blocks within foundation models.

• Shapelets in the era of large language and multimodal models. The nascent integration of shapelets with large language models opens transformative pathways. Research should explore cross-modal quality measures evaluating how shapelets ground temporal reasoning in language models (e.g., via attention-map analysis), or facilitate multimodal alignment in video-text systems. As large language models absorb time-series data, developing “prompt-aware” shapelet metrics that optimize for human-artificial intelligence (AI) collaboration will be critical for scientific discovery and industrial.