Persistent homology classification algorithm

Classification algorithms

Classification is the process of identifying, recognizing, grouping, and understanding new observations into categories or classes (Alpaydin, 2014). A training dataset is composed of ( $n+1$)-dimensional data points which are split into an $n$-dimensional input vector often called attributes and into a one-dimensional output category or class. A dataset can be univariate, bivariate, or multivariate in nature, whereas the attributes are quantifiable properties that can be categorical, ordinal, integer-valued, or real-valued. A classification algorithm or classifier is a procedure that performs classification tasks. The term classifier may also refer to the mathematical function that maps input attributes to an output category.

Classification algorithms have found a wide range of applications in domains such as computer vision, speech recognition, biometric identification, biological classification, pattern recognition, document classification, and credit scoring (Abiodun et al., 2018). Classification problems can be categorized as binary or multi-class. Binary classification entails allocating an observation to one of two distinct categories, whereas multi-class classification involves assigning an observation to one of more than two distinct classes.

Numerous classification algorithms have been developed since the rise of artificial intelligence. While several of these classifiers are specifically developed to solve binary classification problems, some algorithms can be used to solve binary and multi-class classification problems. Many of these multi-class classifiers are extensions or combinations of one or more binary classifiers.

The no-free lunch theorem proved by David Wolpert and William Macready implies that no learning or optimization algorithm works best on all given problems (Wolpert, 1996; Wolpert & Macready, 1997). A classifier can be chosen depending on the type of data at hand. Since then, many state-of-the-art classifiers have been developed. Some of the most commonly used classifiers are logistic regression, Naive Bayes classifier, linear discriminant analysis, support vector machines, k-nearest neighbor, decision trees, and neural networks.

A classification algorithm is a linear classifier if it uses a linear function or linear predictor that assigns a score to each category $k$ based on the dot product of a weight vector and a feature vector. The linear predictor is given by the score function, $Score(X_i,k)={\beta }_k{X}_i$, where $X_i$ is the feature vector for the observation $i$, and ${\beta }_k$ is the weight vector corresponding to category $k$. Observation $i$ is mapped by the linear predictor to the category $k$ with the highest score function ${\beta }_k{X}_i$. Examples of linear classifiers include logistic regression, support vector machines, and linear discriminant analysis (Yuan, Ho & Lin, 2012).

Topological data analysis (TDA)

Data scientists employ techniques and theories drawn from various fields of mathematics, particularly algebraic topology, statistics, information science, and computer science. In mathematics, there is a growing field called topological data analysis, which is an approach that uses tools and techniques from topology to analyze datasets (Chazal & Michel, 2021). In the past two decades, TDA has been applied in various areas of science, engineering, medicine, astronomy, image processing, and biophysics.

Analyzing the shape of data is one of the motivations for using TDA. One of the key tools used in TDA is persistent homology (PH), a method based on algebraic topology that can be used for determining invariant features or topological properties of a space of points that persist across multiple resolutions (Edelsbrunner & Harer, 2008; Carlsson, 2009; Edelsbrunner & Harer, 2010). These derived invariant features are sensitive to small changes in the input parameters which make PH attractive to researchers studying qualitative features of data. PH involves representing a point cloud by a filtered sequence of nested complexes, which are turned into novel representations like barcodes and then interpreted statistically and qualitatively based on persistent topological features obtained (Otter et al., 2017). The following section gives a brief description of how to compute PH and a more detailed discussion of pertinent information about the homology of simplicial complexes and the complete process of computing PH of a point cloud can be found in the Appendix.

Computing persistent homology

Given a finite metric space S, such as a point cloud or a collection of experimental data, it is assumed that S represents a sample from an underlying topological space, such as space T. Describing the topology of T based only on a finite sample S is not a straightforward task. However, PH can be utilized to determine the general shape and topological properties of T while maintaining the robustness of the data.

A point cloud can undergo a filtration process that turns it into a sequence of nested simplicial complexes. This is accomplished by considering a finite number of increasing parameters and recording the sublevel sets that track changes in the topological features of the complexes. More specifically, gradually thickening the points give birth to new topological features, like simplices and $n$-dimensional holes (Zomorodian & Carlsson, 2005). Figure 1 shows an illustration of the filtration process. A more formal and extensive discussion of how to compute the persistent homology of a point cloud is presented in the Appendix.

The appearance (birth) and disappearance (death) of intrinsic topological features, like homology groups and Betti numbers, can be recorded and visualized in many ways. The most frequently used are persistence barcodes or persistence diagrams. The duration or life span of the persisting topological features is essential in analyzing the qualitative and topological properties of a dataset.

Definition 1. Let K be a finite simplicial complex and $K_1\subseteq K_2\subseteq \dots \subseteq K_r=K$ be a finite sequence of nested subcomplexes of K. K is called a filtered simplicial complex and the sequence $\{K_1,K_2,\dots \}$ is called a filtration of K.

The homology of each of the subcomplexes can be computed. For each $p$, the inclusion maps $K_i\to K_j$ induce ${𝔽}_2$-linear maps ${\mathrm{\partial }}_i^j:H_p(K_i)\to H_p(K_j)$ for all $i,j\in \{1,2,\dots ,r\}$ with $i\le j$. It follows from functoriality that ${\mathrm{\partial }}_k^j{}^{\circ }{\mathrm{\partial }}_i^k={\mathrm{\partial }}_i^j$ for all $i\le k\le j$.

Definition 2. Let $K_s$ be a subcomplex in the filtration of the simplicial complex K, or $K_s$ be the filtered complex at time $s$, and $Z_k^s=Ker{\mathrm{\partial }}_k^s$ and $B_k^s=Im{\mathrm{\partial }}_{k+1}^s$ be the $k$-th cycle group and boundary group of $K_s$, respectively. The $k$-th homology group of $K_s$ is $H_k^s=Z_k^s/B_k^s=Ker({\mathrm{\partial }}_k^s)/Im({\mathrm{\partial }}_{k+1}^s)$.

Definition 3. For $p\in \{0,1,2,\dots \}$, the $p$-persistent $k$-th homology group of K given a subcomplex $K_s$ is

$$H_k^{s,p}(K,K_s)=H_k^{s,p}(K)=Z_k^s/(B_k^{s+p}\bigcap Z_k^s)={\displaystyle \frac{Ker({\mathrm{\partial }}_k^s)}{Im({\mathrm{\partial }}_{k+1}^{s+p})\bigcup Ker({\mathrm{\partial }}_k^s)}.}$$

The $p$-th persistent $k$-th Betti number ${\beta }_k^{s,p}$ of $K_s$ is the rank of $H_k^{s,p}(K)$.

Persistent vector spaces and persistence diagrams

Definition 4. A persistent vector space Carlsson (2014) is defined to be a family of vector spaces with linear maps between them, denoted by ${\left\{V_{\delta }\underset{}{\overset{v_{\delta ,{\delta }^{\mathrm{\prime }}}}{⟶}}{V}_{{\delta }^{\mathrm{\prime }}}\right\}}_{{}_{\delta \le {\delta }^{\mathrm{\prime }}\in ℝ}}$, such that

• (1) $dim(V_{\delta })<\mathrm{\infty }$ for each $\delta \in ℝ$,

• (2) there exist ${\delta }_I$, ${\delta }_F\in ℝ$ such that all maps $v_{\delta ,{\delta }^{\mathrm{\prime }}}$ are linear isomorphisms for $\delta ,{\delta }^{\mathrm{\prime }}\ge {\delta }_F$ and for $\delta ,{\delta }^{\mathrm{\prime }}\le {\delta }_I$, and

• (3) there are only finitely many values of $\delta \in ℝ$ such that $v_{\delta -ϵ}\ncong v_{\delta }$

• (4) for any $\delta <{\delta }^{\mathrm{\prime }}<{\delta }^{\mathrm{\prime }\mathrm{\prime }}\in ℝ$, we have ${v_{{\delta }^{\mathrm{\prime }},{\delta }^{\mathrm{\prime }\mathrm{\prime }}}}^{\circ }{v}_{\delta ,{\delta }^{\mathrm{\prime }}}=v_{\delta ,{\delta }^{\mathrm{\prime }\mathrm{\prime }}}$.

Note that there are only finitely many values of $\delta \in ℝ$ for which the linear maps are not isomorphisms. This implies that the indexing set $ℝ$ can be reduced to $\mathbb{N}$, and we may equivalently define a persistent vector space to be an $\mathbb{N}$-induced family ${\left\{V_{{\delta }_i}\underset{}{\overset{v_{i,i+1}}{⟶}}{V}_{i+1}\right\}}_{{}_{i\in \mathbb{N}}}$ such that

• (1) $dim(V_{{\delta }_i})<\mathrm{\infty }$ for each $i\in \mathbb{N}$ and

• (2) there exists $F\in \mathbb{N}$ such that all the $v_{i,i+1}$ are isomorphisms for $i\ge F$.

A persistent vector space can be obtained from a filtered simplicial complex and for each persistent vector space, one may associate a multiset of intervals called a persistence barcode or persistence diagram. A barcode represents each persistent generator with a horizontal line beginning at the first filtration level where it appears and ending at the filtration level where it disappears. Short bars correspond to noise or artifacts in the data, while lengthy bars correspond to relevant topological features. The standard treatment of persistence barcodes and diagrams appears in Edelsbrunner, Letscher & Zomorodian (2002) and Zomorodian & Carlsson (2005).

Presented here is a modern presentation of the definition of persistence diagrams and persistence barcodes (Chowdhury & Mémoli, 2018; Edelsbrunner, Jablonski & Mrozek, 2015).

Let $\mathcal{V}={\left\{V_{{\delta }_i}\underset{}{\overset{v_{i,i+1}}{⟶}}{V}_{i+1}\right\}}_{{}_{i\in \mathbb{N}}}$ be a persistent vector space. Compatible bases $(B_i{)}_{i\in \mathbb{N}}$ for each $V_{{\delta }_i}$, $i\in \mathbb{N}$ can be chosen such that $v_{i,i+1}{|}_{B_i}$ is injective for each $i\in \mathbb{N}$, and $rank(v_{i,i+1})=card(im((v_{i,i+1}{|}_{B_i})\cap B_{i+1}))$, for each $i\in \mathbb{N}$ (Edelsbrunner, Jablonski & Mrozek, 2015). Note that $v_{i,i+1}{|}_{B_i}$ denotes the restriction of $v_{i,i+1}$ to the set $B_i$. Fix such a collection $(B_i{)}_{i\in \mathbb{N}}$ of bases. Then, define

$L:=\{(b,i)|b\in B_i,b\notin Im(v_{i-1,i}),i\in \{2,3,\dots \}\}\cup \{(b,1)|b\in B_1\}.$

For each $(b,i)\in L$, the integer $i$ is interpreted as the birth index of the basis element $b$ or the first index at which a certain algebraic signal appears in the persistent vector space. Now, define a map $l:L\to \mathbb{N}$ as follows

$\ell (b,i):=max\{k\in \mathbb{N}|{v_{k-1,k}}^{\circ }{\dots }^{\circ }{v_{i+1,i+2}}^{\circ }{v}_{i,i+1}(b)\in B_k\}.$

For each $(b,i)\in L$, the integer $\ell (b,i)$ is called the death index or the index at which the signal $b$ disappears from the persistent vector space.

Definition 5. The persistence barcode of $\mathcal{V}$ is defined to be the following multisets of intervals:

$$\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}(\mathcal{V}):=[[{\delta }_i,{\delta }_{j+1})|there exists(b,i)\in L such that {\ell }_{\text{k}}(b,i)=j],$$where the bracket notation denotes taking the multiset and the multiplicity of $[{\delta }_i,{\delta }_{j+1}]$ is the number of elements $(b,i)\in L$ such that $\ell (b,i)=j.$

The elements of $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}(\mathcal{V})$ are called persistence intervals. They can be represented as a set of line segments over a single axis. Equivalently, $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}(\mathcal{V})$ can be visualized as a multiset of points lying on or above the diagonal in ${\overline{ℝ}}^2$ counted with multiplicity, where $\overline{ℝ}$ denotes $[-\mathrm{\infty },+\mathrm{\infty }]$.

Definition 6. The persistence diagram of $\mathcal{V}$ is given by

$$Dgm(\mathcal{V}):=[({\delta }_i,{\delta }_{j+1})\in {\overline{ℝ}}^2|[{\delta }_i,{\delta }_{j+1})\in \mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}(\mathcal{V})],$$where multiplicity of $({\delta }_i,{\delta }_{j+1})\in {\overline{ℝ}}^2$ is given by the multiplicity of $[{\delta }_i,{\delta }_{j+1})\in \mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}(\mathcal{V})$.

Discussion of the robustness and stability of the persistence diagram requires the notion of distance. Given two persistence diagrams, say X and Y, the definition of distance between X and Y is given as follows.

Definition 7. Let $p\in [1,\mathrm{\infty }]$. The $p$-th Wasserstein distance between X and Y is defined as

$$W_p[d](X,Y):=\underset{\varphi :X\to Y}{\overset{}{inf}}{\left[\sum _{x\in X}d{[x,\varphi (x)]}^p\right]}^{1/p}$$for $p\in [1,\mathrm{\infty })$ and as

$$W_{\mathrm{\infty }}[d](X,Y):=\underset{\varphi :X\to Y}{\overset{}{inf}}\underset{x\in X}{sup}d[x,\varphi (x)]$$

for $p=\mathrm{\infty }$, where $d$ is a metric on ${\overline{ℝ}}^2$ and $\varphi$ ranges over all bijections from X to Y.

Normally, $d$ is taken to be $L_q$ where $q\in [1,\mathrm{\infty }]$ and the most commonly used distance function is the Bottleneck distance $W_{\mathrm{\infty }}[L_{\mathrm{\infty }}]$. Equivalently, the bottle distance between persistence diagrams can be defined as follows.

Definition 8. The bottleneck distance between persistent diagrams and more generally, between multisets C, D of points in ${\overline{ℝ}}^2$ is given by

$$d_B(C,D):=inf\{\underset{a\in C}{sup}\Vert a-\varphi (a)\Vert |\varphi :C\cup {\mathrm{\Delta }}^{\mathrm{\infty }}\to D\cup {\mathrm{\Delta }}^{\mathrm{\infty }} is a bijection.\},$$where $\Vert (p,q)-(p^{\mathrm{\prime }},q^{\mathrm{\prime }}){\Vert }_{\mathrm{\infty }}:=max(|p-p^{\mathrm{\prime }}|,|q-q^{\mathrm{\prime }}|)$ for each $p$, $q$, p', and $q^{\mathrm{\prime }}\in ℝ$, and ${\mathrm{\Delta }}^{\mathrm{\infty }}$ is the multiset consisting of each point on the diagonal, taken with infinite multiplicity.

Basis for the classification algorithm

Let I be an index set with K elements, where $K\in \mathbb{N}$. Suppose each $k\in I$ corresponds to a class of points in ${ℝ}^n$. Let $X_k$ be the set of $n$-dimensional points in the Euclidean space belonging to class $k$. Now, suppose there is an $n$-dimensional point ${\alpha }^p=({\alpha }_1^p,{\alpha }_2^p,\dots ,{\alpha }_n^p)\in {ℝ}^n$. The problem of determining which of the K classes of points ${\alpha }^p$ belongs to is a classification problem. Note that the collection of points belonging to the same class has its own topological properties. It is a natural intuition that the inclusion of an additional point ${\alpha }^p$ in a point cloud where it belongs should result in a very small change in its topological properties. What follows is a way to analyze a point cloud and its properties.

Given a point cloud, we can compute its corresponding topological properties in the form of a persistence barcode or a persistence diagram. To do this, we follow the succeeding construction. From here on, this is referred to as the long construction.

For each point cloud $X_k$, $k\in I$, applying the long construction is the same as defining the following;

• (i) the Vietoris Rips complex filtration given by $X_1^k\subseteq X_2^k\subseteq \dots \subseteq X_{m+1}^k$ corresponding to the $X_k$ with fixed $m\in \mathbb{N}$, $ϵ>0$, and the parameter values ${ϵ}_j$’s with ${ϵ}_1<{ϵ}_2<\dots <{ϵ}_{m+1}$ and ${ϵ}_j={\displaystyle \frac{(j-1)ϵ}{m}}$, for each $j\in \{1,2,\dots ,m+1\}$,

• (ii) an $\mathbb{N}$-indexed persistent vector space ${\mathcal{X}}^k={\left\{X_i^k\underset{}{\overset{x_{i,i+1}^k}{⟶}}{X}_{i+1}^k\right\}}_{i\in \mathbb{N}}$ such that

• (1) $dim(X_i^k)<\mathrm{\infty }$ for each $i\in \mathbb{N}$ and

• (2) there exists $m\in \mathbb{N}$ such that all the $x_{i,i+1}$ are isomorphisms for $i\ge m$.

• (iii) chosen compatible bases $(B_i^k{)}_{i\in \mathbb{N}}$ for each respective class $k$ or $X_i^k$, $k\in I$ such that $x_{i,i+1}^k{|}_{B_i^k}$ denote the restriction of $x_{i,i+1}^k$ to the set $B_i^k$ and $rank(x_{i,i+1}^k)=card(im((x_{i,i+1}{|}_{B_i^k})\cap B_{i+1}))$,

• (iv) the set given by $L_k:=\{(b,i)|b\in B_i^k,b\notin im(x_{i-1,i}^k),i\in \{2,3,\dots \}\}\cup \{(b,1)|b\in B_1^k\}$,

• (v) the map defined by ${\ell }_k(b,i):=max\{r\in \mathbb{N}|{x_{r-1,r}}^{\circ }{\dots }^{\circ }{x_{i+1,i+2}}^{\circ }{x}_{i,i+1}(b)\in B_r^k\}$,

• (vi) the persistence barcode given by $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{X}}^k):=[[{\delta }_i,{\delta }_{j+1})|\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e} \mathrm{e}\mathrm{x}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{s}(b,i)\in L_k \mathrm{s}\mathrm{u}\mathrm{c}\mathrm{h} \mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t} {\ell }_k (b,i)=j]$, where the bracket notation denotes taking the multiset and the multiplicity of $[{\delta }_i,{\delta }_{j+1}]$ is the number of elements $(b,i)\in L_k$ such that ${\ell }_k(b,i)=j$, and

• (vii) the persistence diagram given by $\mathbf{D}\mathbf{g}\mathbf{m}({\mathcal{X}}^k):=[({\delta }_i,{\delta }_{j+1})\in {\overline{ℝ}}^2|[{\delta }_i,{\delta }_{j+1})\in \mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}(\mathcal{V})]$, where multiplicity of $({\delta }_i,{\delta }_{j+1})\in {\overline{ℝ}}^2$ is given by the multiplicity of $[{\delta }_i,{\delta }_{j+1})\in \mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{X}}^k)$.

Now, we consider the scenario of adding the $n$-dimensional point ${\alpha }^p=({\alpha }_1^p,{\alpha }_2^p,\dots ,{\alpha }_n^p)\in {ℝ}^n$ to the point clouds. Define $Y_k=X_k\cup \{{\alpha }^p\}$ for each $k\in I$. Apply the long construction for each of these point clouds, that is define the following;

• (i) the Vietoris Rips complex filtration given by $Y_1^k\subseteq Y_2^k\subseteq \dots \subseteq Y_{m+1}^k$ corresponding to the $Y_k$ with fixed $m\in \mathbb{N}$, $ϵ>0$, and the parameter values ${ϵ}_j$’s where ${ϵ}_1<{ϵ}_2<\dots <{ϵ}_{m+1}$ and ${ϵ}_j={\displaystyle \frac{(j-1)ϵ}{m}}$ for each $j\in \{1,2,\dots ,m+1\}$,

• (ii) the $\mathbb{N}$-indexed persistent vector space ${\mathcal{Y}}^k={\left\{Y_i^k\underset{}{\overset{y_{i,i+1}^k}{⟶}}{Y}_{i+1}^k\right\}}_{{}_{i\in \mathbb{N}}}$ similar to that of ${\mathcal{X}}^k$,

• (iii) compatible bases $({\widetilde{B_i}}^k{)}_{i\in \mathbb{N}}$ similar to that of $({B_i}^k{)}_{i\in \mathbb{N}}$,

• (iv) the set ${\tilde{L}}_k$ similar to that of $L_k$,

• (v) the map ${\tilde{\ell }}_k$ similar to that of ${\ell }_k$

• (vi) the persistence barcode or multiset given by $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{Y}}^k)$, and

• (vii) the persistence diagram given by $Dgm({\mathcal{Y}}^k)$.

Finally, classification is done by choosing the class $k$ for which the change in the topological properties between $X_k$ and $Y_k$ is minimum. This is inspired by the following observations.

Theorem 1. Let I be a finite index set. Suppose that for each $k\in I$, $X_k$ is the set of points in ${ℝ}^n$ belonging to class $k$, and that $X_i\cap X_j=\mathrm{\varnothing }$ for each $i\ne j$. Assuming ${\alpha }^p=({\alpha }_1^p,{\alpha }_2^p,\dots ,{\alpha }_n^p)\in {ℝ}^n$ is a point identical to one of the points in $X_p$, for an element $p\in I$. Then, for each $k\in I$,

$$d_B(Dgm({\mathcal{X}}^p),Dgm({\mathcal{Y}}^p))\le d_B(Dgm({\mathcal{X}}^k),Dgm({\mathcal{Y}}^k)),$$where ${\mathcal{X}}^k$, and ${\mathcal{Y}}^k$ are the $\mathbb{N}$-indexed persistent vector spaces corresponding $X_k$ and $X_k\cup \{{\alpha }^p\}$, respectively, for each $k\in I$.

Proof. Let ${\alpha }^p\in X_p$ be an $n$-dimensional point where $p\in I$. Choose any element $k\in I$ such that $k\ne p$. Then, $X_k\cap X_p=\mathrm{\varnothing }$. Apply the long construction for $X_k$, $Y_k=X_k\cup \{{\alpha }^p\}$, $X_p$, and $Y_p=X_p\cup \{{\alpha }^p\}$. From the long constructions leading to the computation of persistence diagrams, we can define the following

• (i) the Vietoris Rips complex filtrations associated to $X_k$, $Y_k$, $X_p$ and $Y_p$, with respect to some $m\in \mathbb{N}$, $ϵ>0$, and the parameter values ${ϵ}_j$’s with ${ϵ}_1<{ϵ}_2<\dots <{ϵ}_{m+1}$ and ${ϵ}_j={\displaystyle \frac{(j-1)ϵ}{m}}$ for each $j\in \{1,2,\dots ,m+1\}$,

• (ii) the $\mathbb{N}$-indexed persistent vector spaces ${\mathcal{X}}^k$, ${\mathcal{Y}}^k$, ${\mathcal{X}}^p$, and ${\mathcal{X}}^p$,

• (iii) respective compatible bases $({B_i}^k{)}_{i\in \mathbb{N}}$, $({B_i}^p{)}_{i\in \mathbb{N}}$, $({\widetilde{B_i}}^k{)}_{i\in \mathbb{N}}$, and $({\widetilde{B_i}}^p{)}_{i\in \mathbb{N}}$,

• (iv) the sets $L_k$, $L_p$, ${\tilde{L}}_k$, and ${\tilde{L}}_p$,

• (v) the maps ${\ell }_k$, ${\ell }_p$, ${\tilde{\ell }}_k$, and ${\tilde{\ell }}_p$,

• (vi) the persistence barcodes or multisets given by $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{X}}^k)$, $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{Y}}^k)$, $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{X}}^p)$, and $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{Y}}^p)$, and

• (vii) the persistence diagrams given by $Dgm({\mathcal{X}}^k)$, $Dgm({\mathcal{Y}}^k)$, $Dgm({\mathcal{X}}^p)$, and $Dgm({\mathcal{Y}}^p)$.

Since ${\alpha }^p\in X_p$, then ${\alpha }^p\notin X_k$, $X_p=Y_p$, and $Y_k$ has one more element than $X_k$. Also, ${\alpha }^p\in {\widetilde{B_1}}^k$ and ${\alpha }^p\notin {B_1}^k$. Note that there is at least one $({\alpha }^p,i)\in \widetilde{L_k}$ with $i$ at least the first index 1. Then $\widetilde{{\ell }_k}({\alpha }^p,i)\ge 1$, by the definition of $\widetilde{{\ell }_k}:\widetilde{L_k}\to \mathbb{N}$. That is, the death index corresponding $({\alpha }^p,i)$ must be greater than 1.

This implies that $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{Y}}^k)$ will have one more element than $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{X}}^k)$. This means that $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{Y}}^k)$ has one unmatched point with respect to the pairing with $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{X}}^k)$. Then, the computation of $d_B(Dgm({\mathcal{Y}}^k),Dgm({\mathcal{X}}^k))$ boils down to the optimal distance between the matched points or the distance between the unmatched point to the diagonal. In either case, this is greater than zero.

Now, since $X_p=Y_p$, then $d_B(Dgm({\mathcal{Y}}^p),Dgm({\mathcal{X}}^p))=0$.

Hence, for each $k\in I$, $d_B(Dgm({\mathcal{X}}^k),Dgm({\mathcal{Y}}^k))\ge d_B(Dgm({\mathcal{X}}^p),Dgm({\mathcal{Y}}^p))=0.$

Theorem 2. Let I be a finite index set. Suppose that for each $k\in I$, $X_k$ is the set of points in ${ℝ}^n$ belonging to class $k$, and that $X_i\cap X_j=\mathrm{\varnothing }$ for each $i\ne j$. Assume ${\alpha }^p\in {ℝ}^n$ is the new data point and ${\alpha }^p\notin X_k$ for each $k\in I$. Let $\alpha {}^{q^{\ast }}$ be a point in $X_{q^{\ast }}$, where $q^{\ast }\in I$, such that $\alpha {}^{q^{\ast }}$ is the closest point to ${\alpha }^p$ among all points in $\bigcup _{k=1}^K{X}_k$. If the long construction is applied to $X_{q^{\ast }}$ and $Y_{q^{\ast }}=X_{q^{\ast }}\cup \{{\alpha }^p\}$, and to $X_k$ and $Y_k=X_k\cup \{{\alpha }^p\}$, for each $k\in I$, $k\ne q^{\ast }$, and the sets $L_k$, $L_{q^{\ast }}$, ${\tilde{L}}_k$, and ${\tilde{L}}_{q^{\ast }}$ are limited to elements with birth index of 1, then for each $k\in I$,

$$d_B(Dgm({\mathcal{X}}^{q^{\ast }}),Dgm({\mathcal{Y}}^{q^{\ast }}))\le d_B(Dgm({\mathcal{X}}^k),Dgm({\mathcal{Y}}^k)),$$where ${\mathcal{X}}^k$ and ${\mathcal{Y}}^k$ are the $\mathbb{N}$-indexed persistent vector spaces corresponding to $X_k$ and $X_k\cup \{{\alpha }^p\}$, respectively, for each $k\in I$.

Proof. Let I be a finite index set. Suppose that for each $k\in I$, $X_k$ is the set of points in ${ℝ}^n$ belonging to class $k$, and that $X_i\cap X_j=\mathrm{\varnothing }$ for each $i\ne j$. Assume ${\alpha }^p\in {ℝ}^n$ is the new point and ${\alpha }^p\notin X_k$ for each $k\in I$. For each $k\in I$, let ${\alpha }_k^q$ be the element in $X_k$ which is closest to ${\alpha }^p$. Suppose that $\alpha {}^{q^{\ast }}$ is the closest element to ${\alpha }^p$ among any ${\alpha }_k^q$, where $k\in I$.

Without loss of generality, choose any element $k\in I$ such that $k\ne q^{\ast }$. Then, $X_k\cap X_{q^{\ast }}=\mathrm{\varnothing }$. Apply the long construction for $X_k$, $Y_k=X_k\cup \{{\alpha }^p\}$, $X_{q^{\ast }}$ and $Y_{q^{\ast }}=X_{q^{\ast }}\cup \{{\alpha }^p\}$. From the long constructions leading to the computation of persistence diagrams, we define the following;

• (i) the Vietoris Rips complex filtrations associated to $X_k$, $Y_k$, $X_{q^{\ast }}$and $Y_{q^{\ast }}$, with respect to some $m\in \mathbb{N}$, $ϵ>0$, and the parameter values ${ϵ}_j$’s with ${ϵ}_1<{ϵ}_2<\dots <{ϵ}_{m+1}$ and ${ϵ}_j={\displaystyle \frac{(j-1)ϵ}{m}}$, for each $j\in \{1,2,\dots ,m+1\}$,

• (ii) the $\mathbb{N}$-indexed persistent vector spaces ${\mathcal{X}}^k$, ${\mathcal{Y}}^k$, ${\mathcal{X}}^{q^{\ast }}$, and ${\mathcal{X}}^{q^{\ast }}$,

• (iii) respective compatible bases $({B_i}^k{)}_{i\in \mathbb{N}}$, $({B_i}^{q^{\ast }}{)}_{i\in \mathbb{N}}$, $({\widetilde{B_i}}^k{)}_{i\in \mathbb{N}}$, and $({\widetilde{B_i}}^{q^{\ast }}{)}_{i\in \mathbb{N}}$,

• (iv) the sets $L_k$, $L_{q^{\ast }}$, ${\tilde{L}}_k$, and ${\tilde{L}}_{q^{\ast }}$ whose elements are restricted to those with birth index of 1,

• (v) the maps ${\ell }_k$, ${\ell }_{q^{\ast }}$, ${\tilde{\ell }}_k$, and ${\tilde{\ell }}_{q^{\ast }}$,

• (vi) the persistence barcodes or multisets given by $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{X}}^k)$, $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{Y}}^k)$, $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{X}}^{q^{\ast }})$, and $\mathbf{P}\mathbf{e}\mathbf{r}\mathbf{s}({\mathcal{Y}}^{q^{\ast }})$, and

• (vii) the persistence diagrams given by $Dgm({\mathcal{X}}^k)$, $Dgm({\mathcal{Y}}^k)$, $Dgm({\mathcal{X}}^{q^{\ast }})$, and $Dgm({\mathcal{Y}}^{q^{\ast }})$.

For each $k\in I$, define ${\alpha }_k^q\in {ℝ}^n$ to be the point in $X_k$ which is closest to ${\alpha }^p$. Note that limiting the basis elements to those with birth index of 1 means that only the 0-dimensional topological properties or the connectedness of the simplices are considered. Also, for each $k\in I$, $X_k$ and $Y_k$ differ only by a single point, the point ${\alpha }^p$. By these, the 0-dimensional hole corresponding to ${\alpha }^p$ in the filtration of $Y_k$ disappears faster if ${\alpha }_k^q$ is closer to ${\alpha }^p$. Moreover the computation of $d_B(Dgm({\mathcal{X}}^k),Dgm({\mathcal{Y}}^k))$ is reduced to the computation of ${\tilde{\ell }}_k({\alpha }^p,1)={\displaystyle \frac{1}{2}\Vert {\alpha }^P-{\alpha }_k^q\Vert }$ for each $k\in I$. Since ${\alpha }^{q^{\ast }}$ is closest to ${\alpha }^p$ among all the ${\alpha }_k^q$’s, then this implies that ${\tilde{\ell }}_{q^{\ast }}({\alpha }^{q^{\ast }},1)\le {\tilde{\ell }}_k({\alpha }_k^q,1)$, for any $k\in I$, where $({\alpha }^{q^{\ast }},1)\in ({\widetilde{B_i}}^{q^{\ast }}{)}_{i\in \mathbb{N}}$ and $({\alpha }_k^q,1)\in ({\widetilde{B_i}}^k{)}_{i\in \mathbb{N}}$. This completes the proof.

Illustration 1. Consider the point clouds labelled as $X_1$, $X_2$, and $X_3$, and the point P in the 2-dimensional space as presented in Fig. 2. Suppose that the point P has to be categorized into one of the point clouds.

For each $i\in \{1,2,3\}$, define $Y_i=X_i\cup \{P\}$ and compute the persistent homologies of $X_i$ and $Y_i$. Presented in Fig. 3 are the respective persistence barcodes corresponding to the 0-dimensional holes in the filtration of the $X_i$’s and $Y_i$’s. Note that each pair of $X_i$ and $Y_i$ differ only by a single point, the point P. Since only the 0-dimensional holes are considered here, then the persistence barcode for $Y_i$ has one more bar than the persistence barcode for $X_i$, for each $i\in \{1,2,3\}$. These differences, represented by the green bars in Fig. 3, are indicative of the shortest distance of P from the point clouds. The life span of the 0-dimensional hole corresponding to point P in the filtration of $Y_i$ is the shortest when $X_i$ is the closest point cloud to P. For each $k\in \{1,2,3\}$,

$$d_B(Dgm({\mathcal{X}}^3),Dgm({\mathcal{Y}}^3))\le {\tilde{\ell }}_k(P,1)={\displaystyle \frac{1}{2}\Vert P-{\alpha }_k^q\Vert ,}$$where ${\alpha }_k^q$ is the point in $X_k$ that is closest to P.

Advances in the fusion of persistent homology and machine learning

Computation of persistent homology (PH) has been applied to a variety of fields, including image analysis, pattern comparison and recognition, network analysis, computer vision, computational biology, oncology, and chemical structures. Some examples can be found in the works of Charytanowicz et al. (2010), Goldenberg et al. (2010), Nicolau, Levine & Carlsson (2011), Xia & Wei (2014), Giansiracusa, Giansiracusa & Moon (2019), and Ignacio & Darcy (2019). Furthermore, advances in the different aspects of computing PH have been increasing at a very rapid rate. Various software have also been developed for computing persistent homology. These software packages include JavaPlex, Perseus, Dipha, Dionysus, jHoles, GUDHI (Geometry Understanding in Higher Dimensions), Rivet, Ripser, PHAT (Persistent Homology Algorithm Toolkit), and R-TDA (R package for Topological Data Analysis) (Otter et al., 2017; Pun, Lee & Xia, 2022). On the other hand, machine learning has been in development as early as 1950s, but it was not until the 1990s that a shift from a knowledge-driven approach to a data-driven approach in studying machine learning took place. It was also during this time when support vector machines and neural networks became very popular. Persistent homology, which has only been around for a decade, has also received attention in recent years.

A direct exposition of the use of machine learning and persistence barcodes was used by Giansiracusa, Giansiracusa & Moon (2019) in solving a fingerprint classification problem. They also showed that better accuracy rates can be achieved when applying topological data analysis to 3-dimensional point clouds of oriented minutiae points. Chung, Cruse & Lawson (2020) used persistence curves, rather than barcodes or diagrams, to analyze time series classification problems. Ismail et al. (2020) used PH-based machine learning algorithms to predict next day direction of stock price movement. Hofer et al. (2017) incorporated topological signatures to deep neural networks and performed classification experiments on 2D object shapes and social network graphs. Gonzalez-Diaz, Gutiérrez-Naranjo & Paluzo-Hidalgo (2020) used PH to define neural networks by incorporating the concept of representative datasets. In their study, they used persistence diagrams in choosing points that would be included in the representative dataset. Chen et al. (2019) leveraged topological information to regularize the topological complexity of kernel classifiers by incorporating a topological penalty, while Pokorny, Hawasly & Ramamoorthy (2014) used topological approaches to improve trajectory classification in robotics. Edwards et al. (2021) introduced TDAExplore, a machine learning image analysis pipeline based on topological data analysis. TDAExplore can be used to classify high-resolution images and characterize which image regions contribute to classification. PH has also been found useful in unsupervised learning and clustering tasks. Islambekov & Gel (2019) presented an unsupervised PH-based clustering algorithm for space-time data. They evaluated the performance of their proposed algorithm on synthetic data and compared it to other well-known clustering algorithms such as K-means, hierarchical clustering, and DBSCAN (Density-based spatial clustering of applications with noise).

Pun, Lee & Xia (2022) published a survey of PH-based machine learning algorithms and their applications. They presented ways how to use persistent homologies to improve machine learning algorithms such as support vector machines, tree-based methods, and artificial neural networks. The strategies require extracting topological features from individual data points and adding them as new attributes to the data points. The classification algorithms are then modified in such a way that they can utilize the additional attributes to classify the data points more accurately. These strategies of modifying learning algorithms by incorporating topological signatures as additional features to the data points are very effective when the data under scrutiny contains high-dimensional points and when points belonging to the same class share common topological properties. This means that the improved algorithm can be used to group together data points with similar shapes or topological properties. On the other hand, these strategies may not work when the dataset under consideration contains data points in a Euclidean space, when each of the points in the dataset has very low dimensions, or if data points from the same class do not share the same topological properties.

On the contrary, the objective of this study was to develop a supervised classification algorithm for dealing with classification problems involving points in a Euclidean space, datasets that can be separated into classes using hyperplanes, or when points from the same class have a certain level of proximity with each other. The proposed classifier in this study takes substantial use of persistent homologies and topological signatures associated with the different classes of points in the dataset instead of the individual data points’ persistent homologies and topological features.

Interpretation of a point cloud’s persistent homology

Let Z be a point cloud. Computing the persistent homology of Z means letting Z undergo filtration and recording the topological properties of Z that persist across multiple resolutions. The result of computing the persistent homology of Z or the summary of the topological properties of the space underlying Z can be visualized using persistence barcodes and persistence diagrams as shown in Definitions 5 and 6. See the Appendix for an illustration. Moreover, it can also be represented by an $n_i\times 3$ matrix and it will be denoted by $\mathcal{P}(Z)$ in this section. The number of rows of $\mathcal{P}(Z)$, $n_i$, is the number of topological features or $n$-dimensional holes detected in the filtration of Z. The 0-dimensional holes are the connected components, 1-dimensional holes refer to loops, 2-dimensional holes represent voids, and so on. One can limit the maximum dimension that can be detected in the filtration of Z. The first column entry of the $i$-th row of $\mathcal{P}(Z)$ indicates the dimension of $i$-th topological feature in the filtration of Z. The second and third column entries in the $i$-th row of $\mathcal{P}(Z)$ give the birth and death time of the $i$-th topological feature, respectively. The birth and death times refer to the filtration steps when the topological feature appears and disappears, respectively.

Let $\mathcal{P}(Z)$ be the persistent homology of a point cloud Z. Although it can be visualized as a persistence diagram, it denotes an $n_i\times 3$ matrix, where $n_i$ is the number of holes, of various dimensions, which are detected in the complex filtration.

Setting of maximum parameter value

The maximum scales, denoted by maxsc, must be fixed. The scale here refers to the size of the parameter values that will be used in the filtration and the computation of the persistent homology of a point cloud. The suitable value for maxsc depends on the data at hand and it can be chosen to be equal to half of the maximum distance between any two points in the point cloud.

Training and classification

Assume that X is a training dataset consisting of ( $n+1$)-dimensional data points categorized into $k$ classes. Each data point’s initial $n$ entries contain its attributes, while the ( $n+1$)-th entry gives its label or classification. Suppose that $X=X_1\cup X_2\cup \dots \cup X_k$, where $X_1$,…, $X_k$ are the $k$ classes of data points, and $m_i$ is the number of elements in $X_i$. Each training point cloud $X_i$ can be viewed as an ( $m_i$) $\times$ ( $n+1$) matrix with each row representing a point belonging in $X_i$.

Suppose that a new $n$-dimensional point, denoted by $\alpha$, needs to be classified. To begin, analyze the different classes in the training set or describe the topological properties of each class in X by computing $\mathcal{P}(X_i)$ for each $i\in \{1,\dots ,k\}$. Then, measure the effect of including $p$ in each class in X. Define $Y_i=X_i\cup \{\alpha \}$ and compute $\mathcal{P}(Y_i)$ for each $i\in \{1,\dots ,k\}$. Record the changes in the persistent homology between $X_i$ and $Y_i$ for each $i\in \{1,\dots ,k\}$. Specifically, record the change from $\mathcal{P}(X_i)$ to $\mathcal{P}(Y_i)$ for each $i\in \{1,\dots ,k\}$. Lastly, identify the class that is least impacted by the inclusion of $\alpha$. This procedure follows the long construction defined in the basis for the classification algorithm.

Score function for PHCA

The use of persistent homology in the development of the score function that will act as the linear predictor for PHCA is described here. Consider a training set categorized into two classes, A and B. Suppose $\alpha$ is a point that needs to be classified and that $\alpha$ belongs to either A or B only, as shown in Fig. 4. The proposed PHCA works by computing the persistent homology of A, B, $A\cup \{\alpha \}$, and $B\cup \{\alpha \}$. Afterward, measure the changes in the topological features from A to $A\cup \{\alpha \}$ and from B to $B\cup \{\alpha \}$. Suppose that point $\alpha$ is closer to point cloud A than point cloud B, then the change in the persistent homology of A to $A\cup \{\alpha \}$ will be lower than the change in the persistent homology from B to $B\cup \{\alpha \}$. This is because the closer a point $\alpha$ to a point cloud, say Z, the birth, and death of some topological features of $Z\cup \{p\}$, particularly the 0-dimensional holes, will come earlier. That is, the early appearance or disappearance of topological features translates to a shorter life span of topological features in the filtration of the complexes. This procedure which is based on Theorems 2 and 3 works particularly well when the point clouds for different classes are disjoint from one another or when a point under consideration is close to a particular point cloud. In light of these, the score function used for PHCA is computed in the following manner.

Recall that we have defined $Y_i=X_i\cup \{\alpha \}$ for each $i\in \{1,\dots ,k\}$, and that $\mathcal{P}(Y_i)$ is the persistent homology of $Y_i$ for each $i\in \{1,\dots ,k\}$. The score function $Score(Y_i)$ is computed as the difference between the sum total of the lifespan of all the topological features in $\mathcal{P}(Y_i)$ and the sum total of the lifespan of all the topological features in $\mathcal{P}(X_i)$. This is equivalent to the difference between the sum of the entries of the third column of $\mathcal{P}(X_i)$ and the sum of the entries of the third column of $\mathcal{P}(Y_i)$. More explicitly, the score function value is computed as

(1) $$Score(X_i)=|\sum _{a\in Y_i}\ell (a,1)-\sum _{b\in X_i}\ell (b,1)|,$$where $\ell$ is the map that sends a basis element with birth index of 1 to its death index.

Finally, the new data point $\alpha$ is classified under class $j$ if $Score(X_j)\le Score(X_i)$ for all $i\in \{1,\dots ,k\}$. Algorithm 1 gives the pseudocode for the persistent homology classification algorithm (PHCA).

Otter et al. (2017) established that computing the persistent homology of a point cloud using the standard algorithm has cubic complexity in terms of the number of simplices in the worst-case scenario. This, combined with the fact that PHCA is a linear classifier, means that the proposed classifier can be used to perform classification tasks in polynomial time.

Iris plant dataset

The iris plant dataset composed of 150 observations created by Fisher (1936) was obtained from the UCI Machine Learning Repository (Dua & Graff, 2017). This is one of the commonly used datasets in pattern recognition. The dataset is divided into three categories: iris setosa, iris versicolour, and iris virginica. Each category is comprised of 50 data points, with four attributes namely sepal length, sepal width, petal length, and petal width (expressed in centimeters).

Table 1 shows the performance of PHCA and the five major classification algorithms in terms of precision and recall per class. Moreover, Table 2 shows the mean performance of the six methods in terms of accuracy, specificity per class, and F1 score per class. Barplots of these results are presented in Fig. 5. LDA, KNN, and PHCA got an accuracy of $96.67\mathrm{\%}$, while CART, SVM and RF obtained an accuracy of $93.33\mathrm{\%}$. Table 3 shows a summary of the Nemenyi test results and the $p-values$ implies that there is no significant difference in the performance of PHCA and the other methods in terms of the mean performance metrics.

Wheat seeds dataset

The wheat seeds dataset was created by Charytanowicz et al. (2010) at the Institute of Agrophysics of the Polish Academy of Sciences in Lublin. It is available at the UCI Machine Learning Repository (Dua & Graff, 2017). The dataset is composed of 210 observations, which is divided equally into three categories: kama, rosa, and canadian wheat variety. Each observation is characterized by seven attributes, namely, area, perimeter, compactness, length of kernel, width of kernel, asymmetry coefficient, and length of kernel groove. All of these attributes are real-valued and continuous.

The performance of PHCA and the five major classification algorithms in terms of precision and recall per class is shown in Table 4. Furthermore, Table 5 compares the six methods in terms of accuracy, specificity per class, and F1 score per class. Barplots of these results are presented in Fig. 6. For the wheat seeds dataset, PHCA got the third highest accuracy, which is at $90.95\mathrm{\%}$. Notice that PHCA got the least performance in some metrics per class, but these were offset when PHCA got the highest performance in terms of precision for class 3, recall for class 1, specificity for class 3, and F1 score for class 3. Moreover, the results of the Nemenyi test, which can be found in Table 6, show that there is no significant difference between the performance of the six methods in terms of the mean performance metrics.

Social network ads dataset

The social network ads dataset was created by Raushan (2017) and was retrieved from the Kaggle repository. The dataset is composed of 400 data points and is comprised of uneven number of observations per class. There are 143 data points for Class 1 and 257 data points for Class 2. Each data point has two attributes, age and estimated salary, and a class label, whether a customer purchased a product or not.

Table 7 shows the performance of PHCA and the five major classification algorithms in terms of precision, recall, specificity, F1 score, and accuracy. Barplots of these results are presented in Fig. 7. For this binary classification problem, PHCA got the highest recall at $86.89\mathrm{\%}$. PHCA also got the second-highest F1 score and accuracy, which are at $88.55\mathrm{\%}$ and $85\mathrm{\%}$, respectively. The results of the Nemenyi test, which is summarized in Table 8, show that there is no significant difference between the performance of PHCA and the other classification algorithms in terms of the mean performance metrics. Moreover, the mean rank difference of 18 and the $p-value$ of 0.0155 between KNN and SVM shows that SVM performed significantly better than KNN.